I want to find articles published in a specific journal e.g. “Nature Climate”. I can search for the term “nature climate” but then I find all sorts of documents where it is listed in references. I just want to find those documents that were published in “nature climate”. How do I do that? or can I?
Many thanks
Well, I think that unless there is some meta data attached to these Nature Climate documents (PDF’s?) then you need to inspect one of the documents and find the unique pattern of text which you can then search for.
I just looked at the PDF for Living in the overshoot age | Nature Climate Change and I notice at top the url Living in the overshoot age | Nature Climate Change which suggests to me that if you search for text inside the files “doi.org/10.1038” which i believe is unique to this journal, you’ll find what you are looking for.
I don’t have access to any of the pay-walled articles so I can’t test this further for you.
Others may have other ideas.
You may be able to narrow your search by using restrictions for specific capitalization – Nature Climate. Unfortunately, I do not see an easy way to set this restriction in global or smart searches.
–
JJW
Just thinking out loud here. If you want to have a certain solution, your article needs to have something that identifies the origin. Two thinks I am thinking off are:
As @rmschne pointed out, you need to find the searchable data.
You also need to define things more clearly and specifically in your search terms. In your mind it’s "a specific journal e.g. “Nature Climate”. That does not mean that’s what your computer knows about it. For example:
name: search prefix when searching for things by name. However, you need to know about the document names, e.g., the PDF document may be from that source, but the name may be something like nc-01012026-v1.pdf.nature climate is not the same as searching for "nature climate". The first matches nature AND climate, no matter where the words are, relative to each other. "nature climate" matches the phrase specifically.mdjournal:nature climate. Here is an example showing the Data inspector, the raw and advanced search queries, and a matching document with the custom metadata field as a List view column…Perhaps just search for the ISSN number? I suspect it’s embedded in every issue. Although perhaps not in specific papers.
Thanks to all for their responses. The suggestion of using Bibliographic Metadata is closest to the mark. I am aware that nature climate is not the same as “nature climate” and have been careful to search for the exact journal name.
I realize that the doi also must have some link to the journal name, and in my case I found what I wanted, however in many instances, references also contain ‘doi’s and I only want the first instance, ideally the one on the title page. So, is there a way to search for the specific part of the doi, or the journal name AND only on the first page of the document.
thus find Journal name == “Nature Climate” (or doi == “something”) AND document page == 1
Don
I am trying the Download Bibliographic metadata rule and when it works, it works well; but it often stops after a few minutes and there are lots of articles that have not been abstracted. I am not using DT when this process is ongoing, but I am using my computer.
There are ~31000 documents to be abstracted. Is that too many? If so, how can I do the process on subsets of the database?
Thanks
You definitely should not be queueing up 30,000+ articles. That is a massive number to try and process in one shot. This is making web queries for each document so you’re talking network and resource consumption.
You can duplicate this smart rule and add a filtering criterion and an action to apply the attribute. For example, I added Label is not Done as a criterion then the Change Label action to add the Done label. This way the documents won’t be matched after they’re run through the smart rule.
Now you can grab a handful and either drag and drop them on the smart rule or use Tools > Apply Rules and choose your copied smart rule, e.g.,…
Could you address where the search criteria can be set to choose between ignoring CapiTaliZation or not? I see only one place for an explicit toggle, and it is not in Smart Rules. Is this a case for example of using “Nature” versus “nature” (both in quotes)??
–
JJW
In addition to the method proposed by @BLUEFROG, you could create a folder called something like “< Process”, set the smart rule to apply to anything in that folder, and set the smart rule to mark the document as DONE and move the processed document out to a different folder called something like “> Processed”. As you accumulate new files, drop them into the “< Process” folder to have them processed.
–
JJW
A database’s index is not case-sensitive.
Apologies … Perhaps my request was not clear.
Just so I am understanding your answer … We cannot in any way set up a smart search in DT to find content containing “Nature” versus content containing “nature” versus content containing “naTure”?
Even a REGEX method?
I ask out of genuine ignorance, not (yet) ever needing to perform case-sensitive searches on content in DT.
–
JJW
No worries at all!
Correct. The index doesn’t distinguish case, so toolbar searches ignore it, even when the term is quoted.
Also, the toolbar search doesn’t support regex.
However, in-document searches via the Search inspector do allow for case-sensitive searching…
Thanks.
Any thoughts on whether this limitation might be overcome in a future update to DT? Alternatively said, might I make a feature request to expand the capabilities of searching in DT to allow smart searches to be set up using a check box akin to [ ] ignore case or [ ] case-sensitive?
–
JJW
That would be up to development to answer, but the index would easily grow / bloat if case was distinguished. Also, in the case of spam or bad OCR, e.g., Malware, mAlware, maLware, etc. would be independently indexed, increasing bloat in the index.
I quickly realized that one issue was the 31000 number, so I have started to do the process using groups based on file size (e.g. < 100KB, 100-150KB) and on up till I finish. That works well. The suggestion of adding DONE is great and I will try it.
I don’t think file size matters much in this case. The important part is the number of files processed in a batch.
I agree, but it is a convenient way to group files.
Fair enough. The trade offs in refining search function capabilities are an expansion in database size and a commensurate potentially noticeable slowdown in search speed.
In any case, after re-reading, I see that having case-sensitive searching would not solve the OP’s problem entirely. Some of the articles found in a case sensitive search on content: “Nature Climate” would be in articles from other journals that contain citations to the desired journal title. Even the proposed DOI search method could fail because, in some cases, the references have DOI links to their citations. The better approach to meet the OP’s demand is to do be able to a search on the specific bibliography-type meta content journal: Nature Climate. Which, if I might add with due respect, is a process often more efficiently handled using a dedicated bibliography-management software application.
–
JJW
