Download bibliographic metadata

In DT3 beta2, the premise of the new Download Bibliographic Metadata is awesome - I’ve been trying to think of a way to grab that stuff from Zotero, but doing it directly is a great idea.

I have two issues. First, the dependability of the current way of grabbing the DOI seems to depend on the discipline. It works wonderfully on the few medical articles I have, but barely works on any of the psych articles I have. Are there other methods of grabbing them that might work with more disciplines?

Second, the way it currently sets the citation item is not universal. I’m trying to change the script to set the citation metadata to “Author et al. (year)”, but it doesn’t seem to be taking. The change I made was to edit the script from

this: add custom meta data (citation & " " & pubyear) for “citation” to theRecord

to this: add custom meta data (citation & " (" & pubyear & “)”) for “citation” to theRecord

but it doesn’t seem to take.

I’ve also removed the line about setting the title, since I set that myself on import, and that change worked fine.

(Incidentally, is this implementation dependent on purchasing DT3Pro with custom metadata? It’s a nice bonus feature, but is not going to be worth $100 to me.)

I was thinking it would be interesting if it could get the metadata from the attached reference in Bookends. Or perhaps look it up online using the ISBN nr - since most books don’t have a DOI.

Yes, custom metadata is a Pro edition feature.

The following script will collect the metadata from Bookends and do the following:

  • Populate the PDF title and author metadata fields (need to have exiftool installed)

  • Populate three custom metadata fields (abstract, citation and reference) from Bookends which corresponds to the abstract, Bookends citation and a formatted reference (these latter two will need to be changed in the script to reflect your own personal needs).

  • Create a link between Bookends and DEVONthink Pro

  • An important proviso is that the record in DEVONthink is selected as well as the corresponding reference in Bookends

     -- Script to link DEVONthink file and Bookends reference along with custom metadata
     -- Kyle Eggleton April 2019
    
      tell application "Bookends"
         tell front library window
      
      -- Get selected publication 
      set theRefs to selected publication items
      set theRefsNo to count of theRefs
      set theRef to first item of theRefs
      
      -- Error messages	
      if theRefsNo is greater than 1 then error "Select only one item"
      if theRefs is {} then error "Nothing selected in Bookends"
      
      -- Get properties of selected reference
      set theID to id of theRef
      set theCitation to user1 of theRef -- user1 is the Bookends field where BibTex citation is stored
      set theAbstract to abstract of theRef
      set theAbstract to abstract of theRef
      set theAuthor to authors of theRef
      set theTitle to title of theRef
      set theAttachments to attachments of theRef
      set theJournal to journal of theRef
      set theFormattedReference to format theRef using "APA 6th Edition Markdown.fmt" --change to reflect the formatted reference required
      
      -- Only set metadata fields for PDF if there is an attachment
      if theAttachments is equal to "" then error "There is no file attached to the Bookends reference"
      if theAttachments is not equal to "" then
      	set theFile to the first attachment item of theRef
      	set thePath to the path of theFile
      	if theFile is "" then error "The file is missing from the Bookends reference"
      	
      	-- Set the metadata fields for PDF. Need to install exiftool
      	do shell script "/usr/local/bin/exiftool -title=" & quoted form of theTitle & " -author=" & quoted form of theAuthor & " -subject=" & quoted form of theJournal & " -overwrite_original " & quoted form of thePath
      end if
    end tell
    end tell
    
    
    -- Set URL to Bookends and custom metadata in DevonThink
    tell application id "DNtp"
     try
      set theURL to ""
      set theRecord to the content record of think window 1
      if theRecord is {} then error "Nothing selected in DEVONThink Pro"
      set the URL of theRecord to ("bookends://sonnysoftware.com/" & theID) as text
      add custom meta data theAbstract for "abstract" to theRecord
      add custom meta data theCitation for "citation" to theRecord
      add custom meta data theFormattedReference for "reference" to theRecord
      set theURL to reference URL of theRecord
     end try
    end tell
    
    -- Set URL to DEVONThink in Bookends
    
    tell application "Bookends"
    tell front library window
      set user4 of publication item id theID to theURL
    end tell
    
    end tell
1 Like

What I don’t understand is: Where do the bibliographic data is downloaded to? Where do I find them after they are downloaded?

I do not see any metadata downloaded for articles from arxiv.org or other mathematics websites. Could you please give an example of a link where the script works fine ? Just to try and check that I understand correctly how to use the script.

Thanks,
Richard.

 add custom meta data theAbstract for "abstract" to theRecord
  add custom meta data theCitation for "citation" to theRecord
  add custom meta data theFormattedReference for "reference" to theRecord

As noted here, it’s adding custom meta data, which is displayed in Tools > Inspectors > Custom in the Pro and Server editions. These attributes are also shown and defined in Preferences > Data.

1 Like

exiftool is not part of the standard installation on a Mac. You should make a note to people this is a required dependency in your script.

Thanks for sharing your script! Unfortunately I could not get it to work properly over here.
For some reason it is taking data from another reference in Bookends, other than the one I have selected, and throwing it at the selected reference in DT3. I will dig around to see if I can find the cause of this.

There are some assumptions with the script that I didn’t clarify. The first is that DEVONthink is indexing the Bookends folder where attachments are stored. The second is that there is only one attachment in the selected Bookends reference.

Having used several bibliography database programs, I kept returning to one I’ve used for decades now (Endnote). So I use it in conjunction with DevonThink for searching, with DevonThink doing its great job of indexing. The smart group feature allows me to grab all types of attachments from my bibliography database. I’m not aware of an Endnote script that could work for the variety of reference types I use, but I could easily be missing something.

This is fantastic.:smiley: Any possibility of modifying the script to import highlighted text as well, pls?

doclrb

Yes, have a look at the second script on this thread. It will import Bookends notecards.

Great! TY

doclrb

Not sure if it’s useful but here’s a script similar to File > Import > References from Bookends. Contrary to the internal command the script imports only the selection and adds also custom metadata. And as it’s a script it can be customized of course.

-- Import selected Bookends references as rich text including tags & custom metadata

tell application "Bookends"
	tell front library window
		set theRefs to selected publication items
		repeat with theRef in theRefs
			set theID to id of theRef
			set theDOI to doi of theRef
			set thePMID to pmid of theRef
			set theISBN to isbn of theRef
			set theURL to url of theRef
			set theLanguage to language of theRef
			set theDate to publication date string of theRef
			set theCreationDate to date added of theRef
			set theModificationDate to date modified of theRef
			-- set theType to type of theRef
			
			set theTitle to title of theRef
			set theNotes to notes of theRef
			set theKeywords to keywords of theRef
			set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ASCII character 10}
			set theTags to text items of theKeywords
			set AppleScript's text item delimiters to od
			
			set theAuthors to authors of theRef
			set theEditors to editors of theRef
			set thePublisher to publisher of theRef
			set theJournal to journal of theRef
			set theVolume to volume of theRef
			set thePages to pages of theRef
			set theRating to rating of theRef
			set theCitation to user1 of theRef -- user1 is the Bookends field where BibTex citation is stored
			set theAbstract to abstract of theRef
			set theContent to format theRef using "Summary.fmt" as RTF
			set theFormattedReference to format theRef using "APA 6th Edition Markdown.fmt"
			
			(* repeat with theAttachment in attachments of theRef
				set theFile to the first attachment item of theRef
				set thePath to the path of theFile
				if thePath is not "" then
				end if
			end repeat *)
			
			tell application id "DNtp"
				set theRecord to create record with {name:theTitle, type:rtf, content:theContent, rating:theRating, comment:theNotes, tags:theTags, creation date:theCreationDate, modification date:theModificationDate} in current group
				
				set URL of theRecord to ("bookends://sonnysoftware.com/" & theID) as text
				add custom meta data theDOI for "doi" to theRecord
				add custom meta data thePMID for "pmid" to theRecord
				add custom meta data theISBN for "is?n" to theRecord
				add custom meta data theURL for "link" to theRecord
				add custom meta data theLanguage for "language" to theRecord
				add custom meta data theDate for "date" to theRecord
				-- add custom meta data theType for "type" to theRecord
				
				add custom meta data theAuthors for "authors" to theRecord
				add custom meta data theEditors for "editors" to theRecord
				
				add custom meta data thePublisher for "publisher" to theRecord
				add custom meta data theVolume for "volume" to theRecord
				add custom meta data theJournal for "journal" to theRecord
				add custom meta data thePages for "page" to theRecord
				
				add custom meta data theAbstract for "abstract" to theRecord
				add custom meta data theCitation for "citation" to theRecord
				add custom meta data theFormattedReference for "reference" to theRecord
			end tell
		end repeat
	end tell
end tell
2 Likes

Is exiftool something one can trust? Any known problems after it is installed?

Phil Harvey’s exiftool has been around and in development for a long time.

Do note: if you’re creating scripts to share here or elsewhere, you should disclose any such dependencies… or better yet, avoid them and write native methods to do the same thing whenever possible.

Another question. Will this run under Devonthink Pro Office 2? When I try to compile I receive at erro message at ‘custom data’.

TY

doclrb

Will this run under Devonthink Pro Office 2?

No. Custom meta data is a DEVONthink 3 feature.

OK. So how do I upgrade to the Pro 3 Beta from my DevinThink Pro Office license from the app store.

TY

Les