Download bibliographic metadata

There is no “DEVONthink Pro Office license from the app store”. Pro Office is not sold on the App Store (and actually never has been).

We recently blogged about using your customer account. I would suggest starting there, regrading upgrades:

However, remember: DEVONthink 3 is in beta testing meant for testing, finding and reporting bugs. It should not be used in mission-critical or production environments.

When beta testing, it’s always wise to use copies of data or test data. I personally also advocate running DEVONthink 3 on a separate macOS User account to keep things isolated. Your data, your choice - but I always err on the side of caution with peoples’ data.

OK. I stand corrected. I have a valid license and have previously forwarded to sales, but it is not listed in my account, hence my my confusion. In order to upgrade my account I imagine it would have to be listed there, but it is not. So can we start by taking steps to list my license in my account? The appropriate steps should follow from there,

Please go to https://www.devontechnologies.com/support/supportcenter and contact Heike regarding sales and your account. Thanks!

Thanx again for providing the script. I have muddled my way through the DT3 beta upgrade and compiling and inserting the script. When I run it, a highlighted ref is pulled in to DT3 and an ‘Annotations’ subhead is created, but I do not see any highlighted text. I’m so close. What am I missing here? I believe it has something to do with the notecard feature in Bookends. Would really appreciate guidance.

TY

doclrb

GOT IT. One must specifically ‘Add the highlights to Bookends’ and it works ike a charm!

TY

doclrb

Glad it worked out for you

Like the Import > References from Bookends... command, can the script be modified so that it only updates/ignores already created .rtf files instead of adding duplicates? Since I’ve no experience with Apple Script any help is really appreciated.

Thanks.

A simple if ... end if condition around the block creating the data is already sufficient. Just curious but why do you use the script instead of the command?

-- Import selected Bookends references as rich text including tags & custom metadata

tell application "Bookends"
	tell front library window
		set theRefs to selected publication items
		repeat with theRef in theRefs
			set theID to id of theRef
			set theDOI to doi of theRef
			set thePMID to pmid of theRef
			set theISBN to isbn of theRef
			set theURL to url of theRef
			set theLanguage to language of theRef
			set theDate to publication date string of theRef
			set theCreationDate to date added of theRef
			set theModificationDate to date modified of theRef
			-- set theType to type of theRef
			
			set theTitle to title of theRef
			set theNotes to notes of theRef
			set theKeywords to keywords of theRef
			set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ASCII character 10}
			set theTags to text items of theKeywords
			set AppleScript's text item delimiters to od
			
			set theAuthors to authors of theRef
			set theEditors to editors of theRef
			set thePublisher to publisher of theRef
			set theJournal to journal of theRef
			set theVolume to volume of theRef
			set thePages to pages of theRef
			set theRating to rating of theRef
			set theCitation to user1 of theRef -- user1 is the Bookends field where BibTex citation is stored
			set theAbstract to abstract of theRef
			set theContent to format theRef using "Summary.fmt" as RTF
			set theFormattedReference to format theRef using "APA 6th Edition Markdown.fmt"
			
			tell application id "DNtp"
				set theGroup to current group
				set theRefURL to ("bookends://sonnysoftware.com/" & theID) as string
				if not (exists record with URL theRefURL in (database of theGroup)) then
					set theRecord to create record with {name:theTitle, type:rtf, content:theContent, rating:theRating, comment:theNotes, tags:theTags, creation date:theCreationDate, modification date:theModificationDate, URL: theRefURL} in theGroup
					
					add custom meta data theDOI for "doi" to theRecord
					add custom meta data thePMID for "pmid" to theRecord
					add custom meta data theISBN for "is?n" to theRecord
					add custom meta data theURL for "link" to theRecord
					add custom meta data theLanguage for "language" to theRecord
					add custom meta data theDate for "date" to theRecord
					-- add custom meta data theType for "type" to theRecord
					
					add custom meta data theAuthors for "authors" to theRecord
					add custom meta data theEditors for "editors" to theRecord
					
					add custom meta data thePublisher for "publisher" to theRecord
					add custom meta data theVolume for "volume" to theRecord
					add custom meta data theJournal for "journal" to theRecord
					add custom meta data thePages for "page" to theRecord
					
					add custom meta data theAbstract for "abstract" to theRecord
					add custom meta data theCitation for "citation" to theRecord
					add custom meta data theFormattedReference for "reference" to theRecord
					
					(* tell application "Bookends"
						repeat with theAttachment in attachment items of theRef
							set thePath to the path of theAttachment
							if thePath is not "" then tell application id "DNtp" to import thePath to theGroup
						end repeat
					end tell *)
				end if
			end tell
		end repeat
	end tell
end tell

Is that line correct? I’d have expected isbn as the target data.

Sure, it’s the identifier of the default ISBN/ISSN custom metadata.

I gathered that much. But why the question mark in the custom meta data name?

Also, there is
set theURL to url of theRef
at the beginning of the repeat loop. A bit further down, the script says
set theURL to ("bookends://...")
Isn’t that changing the value of theURL set before? And what happens then in
add custom meta data theURL for "link" to theRecord ?
Will “link” become the “bookends://…” URL (which I doubt is intended) or …?

You’re right, fixed.

I’d like to propose an alternative script, written in JavaScript. See below for explanations

(() => {
  const DT = Application("DEVONthink 3");
  const group = DT.currentGroup();
  const groupDB = group.database();
  const BE = Application("Bookends");
  const window = BE.libraryWindows[0];
  /* The fields to get from Bookends. Only those that are used in addCustomMetaData! */
  const bookendFields = ["doi","pmid", "isbn", "url", "language", "publicationDateString", 
    "keywords", "authors", "editors", "publishers", "journal", "volume",
    "pages", "user1", "abstract"];
  /* Loop over all selected Bookends items */
  window.selectedPublicationItems.forEach(item => {
    /* Build the BE URL and skip this item if it's already imported */
    const URL = `bookends://sonnysoftware.com/${item.id()}`;
    if (DT.existsRecordWithURL(URL, {in: groupDB})) return;
    /* Item is new – create a DT record using some of the BE data */
		const record = DT.createRecordWith({
      name:             item.title(), 
      type:             "rtf", 
      content:          BE.format(item, {using: "Summary.fmt", as: "RTF"}), 
      rating:           item.rating(), 
      comment:          item.notes(), 
      tags:             item.keywords().split('\n'), 
      creationDate:     item.dateAdded(), 
      modificationDate: item.dateModified(),
      URL:              URL
    }, {in: groupDB});
    /* Copy content for all BE fields in 'bookendFields' to new record's custom meta data.
       special handling for 'url' and 'user1', since those are named differently in DT */
    bookendFields.forEach(key => {
      if (key === "url") {
        DT.addCustomMetaData(item.url(), {for: "link", to: record});
      } else if (key === "user1") {
        DT.addCustomMetaData(item.user1(), {for: "citation", to: record});
      } else {
        DT.addCustomMetaData(item[key](), {for: key, to: record});
      }
      DT.addCustomMetaData(BE.format(item, {using: "APA 6th Edition Markdown.fmt"}));
    })
  })
})()

In my opinion, this code is “better” because

  • It’s a lot shorter, thus easier to understand (for some, I think),
  • It’s less repetitive, thus easier to read,
  • It uses an array of fields to get from Bookends and use as custom metadata in DT. That makes it easy to add/remove more fields
  • It removes invariants from the loop over the Bookends records (the DT group and this group’s database)
  • It checks if the record already exists before it does any other work and immediately skips to the next Bookends entry in this case.
  • It’s commented :wink:

I think that some of that cannot be done in AppleScript, though. For example, it doesn’t seem to be (directly) possible to access a record’s element using a variable like in

set key to "rating"
set rating to key of theRef

But hoisting the invariants out of the repeat loop is possible (and better, in my opinion). Also, checking for the existence of the record in DT would be possible before retrieving all the data from bookends.

One serious disadvantage, though: I couldn’t test the code, due to a lack of Bookends data. If someone can provide me with test data, I’d love to run the script on them.

1 Like

Even AppleScript can be a lot shorter. But easier to understand? Maybe.

tell application id "DNtp"
	set theGroup to current group
	set theDatabase to database of theGroup
end tell
tell application "Bookends"
	set theMetadataKeys to {"doi", "pmid", "is?n", "link", "language", "date", "authors", "editors", "publisher", "journal", "volume", "page", "citation", "abstract", "reference"}
	set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ASCII character 10}
	repeat with theRef in ((selected publication items of front library window) as list)
		set theRefURL to ("bookends://sonnysoftware.com/" & (id of theRef as string)) as string
		tell application id "DNtp" to set existsRef to exists record with URL theRefURL in theDatabase
		if not existsRef then
			set theMetadataValues to {doi, pmid, isbn, url, language, publication date string, authors, editors, publisher, journal, volume, pages, user1, abstract} of theRef
			set theFormattedReference to format theRef using "APA 6th Edition Markdown.fmt"
			copy theFormattedReference to end of theMetadataValues
			set {theTitle, theNotes, theCreationDate, theModificationDate, theRating, theKeywords} to {title, notes, date added, date modified, rating, keywords} of theRef
			set theContent to format theRef using "Summary.fmt" as RTF
			tell application id "DNtp" to set theRecord to create record with {name:theTitle, type:rtf, content:theContent, rating:theRating, comment:theNotes, tags:text items of theKeywords, creation date:theCreationDate, modification date:theModificationDate, URL:theRefURL} in theGroup
			set i to 1
			repeat with theMetadataKey in theMetadataKeys
				tell application id "DNtp" to add custom meta data (item i of theMetadataValues) for theMetadataKey to theRecord
				set i to i + 1
			end repeat
		end if
	end repeat
	set AppleScript's text item delimiters to od
end tell
1 Like

And this revision avoids almost all add custom meta data calls:

tell application id "DNtp"
	set theGroup to current group
	set theDatabase to database of theGroup
end tell
tell application "Bookends"
	set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ASCII character 10}
	repeat with theRef in ((selected publication items of front library window) as list)
		set theRefURL to ("bookends://sonnysoftware.com/" & (id of theRef as string)) as string
		tell application id "DNtp" to set existsRef to exists record with URL theRefURL in theDatabase
		if not existsRef then
			set theMetadata to {|doi|:doi, |pmid|:pmid, |is?n|:isbn, link:url, |language|:language, |date|:publication date string, |authors|:authors, |editors|:editors, |publisher|:publisher, |journal|:journal, |volume|:volume, page:pages, citation:user1, |abstract|:abstract} of theRef
			set theFormattedReference to format theRef using "APA 6th Edition Markdown.fmt"
			set {theTitle, theNotes, theCreationDate, theModificationDate, theRating, theKeywords} to {title, notes, date added, date modified, rating, keywords} of theRef
			set theContent to format theRef using "Summary.fmt" as RTF
			tell application id "DNtp"
				set theRecord to create record with {name:theTitle, type:rtf, content:theContent, rating:theRating, comment:theNotes, tags:text items of theKeywords, creation date:theCreationDate, modification date:theModificationDate, URL:theRefURL} in theGroup
				set custom meta data of theRecord to theMetadata
				add custom meta data theFormattedReference for "reference" to theRecord
			end tell
		end if
	end repeat
	set AppleScript's text item delimiters to od
end tell
2 Likes

I think that’s cleaner. But there’s still the repetition of theMetadataKeys in theMetadataValues. Which is, IIRC, necessary because it’s not possible to use a variable to access a record’s fields. Therefore, the repeat must include a counter. Otherwise, a single list with all the keys would be sufficient ⇒ data-driven approach, easier to modify, not necessary to repeat one-self. (The later version using |BEkey|:DTkey is a lot better, since it uses only a single data structure).

In general, a shorter text is easier to understand. Compare Thomas Mann to Bild-“Zeitung” :wink: And in my mind, that’s also true for programming languages. The longer a line becomes, the more difficult it is to keep its context in mind. That’s also a reason why I find the continuous using of the in AppleScript code quite irritating (no offense intended): The the is redundant (since there’s only one article in English anyway, so what’s the point of putting it in front of every variable?). Like in (just as an example) theMetadataValues and theMetadataKeys: The important thing comes at the very end, I have to mentally skip over 11 characters before getting to see that we’re talking about Keys vs. Values. But then Java tends to be similarly verbose :wink:

The exception to the “shorter, easier to understand” rule is, of course, extremely condensed code like Array.map(...).filter(…).sort(…)[0]. Which I tend to produce occasionally in JavaScript :frowning:

As to is?n: I assume that’s because there already is an ISBN/ISSN field in the metadata, and for some reason the script doesn’t want to use this, but rather a custom metadata field

Thanks for the script, will try that.

Just curious but why do you use the script instead of the command?

The command places the all authors’ name to the beginning of the title of the .rtf file. I just prefer only to include the reference title from bookends.

Also I would like set my own folder location which is not possible with the current command. One major issue with that is I have the folder named Bookends since I index my Bookends folder in iCloud Drive which causes the folders with the same name - one with there PDFs, the other with the Rich Texts.

How can I tell your script to update the rtf file - currently modifications in Bookends Refs are not reflected in to the rtf that already sent to DT ?

Thanks

That is not the intent of the script. The script creates a document with data from Bookends. It’s not syncing changes between the two.