I have made a few changes to the script to support logging. That is, instead of simply telling you how many sources were added, it will log the exact changes made to a plain text file so you can see exactly what groups are created, note files created, and where they are replicated to:
--Zotero to DEVONThink Pro 1.6
(*
by K. M. Lawson
http://muninn.net/
Please let me know if you find ways to improve on this script and share it with others. Feel free to edit or delete the comments below as needed:
This script was designed for my own uses. I have the following setup and need:
-I often scrape useful citation information from web databases into the Firefox Zotero plugin
-In DEVONThink Pro, in a folder/group called "Sources" I keep a folder for each source
-In each source folder I create a rich text file with some general notes on the source, timeline, and key concepts. I may also keep a PDF of an article and sometimes I have a timeline here in this folder as well.
-When taking notes on the source, I use a "Note on Source" template for DEVONThink of my own creation which makes a new note file in this folder, using the same tags as the main note file. Each fragment or cluster of related notes gets its own file which can then be grouped and organized later as if they were separate note cards.
-I want to automate the process of creating folders and a main note file for every Zotero bibliographic entry.
This script does the following:
-Whenever the script is run, it will get a list of all titles in the Zotero database that aren't attachments
-It will see if a group with the name of the source already exists in DEVONthink Pro. If it does, it moves on to the next entry.
-If there is no such group, one is created
-In this new group a new rich text file is created to be the main note file for that source
-The new rich text file in the group for the source is given tags corresponding to whatever collections and tags it had in Zotero
Notes and Known Issues:
-Please note that if you have a large Zotero database, this script can take a long time to run. You may need to wrap this in "with timeout" clause. I haven't tested this with very large databases.
-Please note: While you can update your list based on new additions to the Zotero database when you like, keep in mind that if you rename the groups in DEVONthink such that they are no long identical to the title in Zotero, this script will assume you do not have an entry for that source and create a new one.
-If you have a Zotero title that is also exactly the same as a collection name or tag, for every source with the collection/tag, you will end up with replicate note files in both its own folder and in the folder with the same name as the tag/collection
-Although this script only reads data from your Zotero database and thus should do it no harm, it is always good to keep good backups of Zotero files. Use the script at your own risk.
-If you move or change the hierarchy of folders in Zotero and run the script again later, it will not delete the old groups or move them (of course, neither will it recreate folders in old locations, so after running simply go through and move and replace the newly created groups (if you have notes in them, just delete them in the old location if you don't)
WAYS TO IMPROVE THIS SCRIPT:
-make a few big SQL calls at the beginning and store all the title, author, and collection data at once instead of making SQL calls to the Zotero database during every loop - will reduce the number of calls and speed up the script.
UPDATED 2010.9.10 1.1
-Makes a copy of the Zotero database and puts it in the tmp file, then deletes this copy when done. Otherwise, an open Firefox will prevent the reading of the file.
UPDATED 2010.9.15 1.5
-Now puts all sources into an "_All" folder (or whatever you call it)
-Puts a replicate of each source into a collection group corresponding to that in Zotero
-Option to not warn about Firefox
-Options to set the formatting of the name of the group and note file and its divider
UPDATED 2010.10.16
-added ability to log all changes made
*)
global groupName
global logFile
global logTitle
--CONFIGURATION - You need to set a few variables to get this to work for you, then put it in the DEVONthink script folder:
--Change the group name to /[group name for your notes]/
set groupName to "/Sources/"
--Do you wish the collections to also be assigned as regular tags to the items in addition to created as groups? If so, set this to true
set collectionTags to false
--If you don't want the folder with all the sources to be called _All change it here:
set allName to "_All"
--If you want to be warned before Firefox is quit, set this to true:
set firefoxWarn to false
--Change the location below to the exact location of your zotero.sqlite. You can find it in Zotero advanced preferences "Show Data Directory"
set zoteroDatabase to "'~/Library/Application Support/Firefox/Profiles/[profile]/zotero/zotero.sqlite'"
--Set the type of group name you want
-- 1 = just the title of the work (Note: if you have multiple works with same title, only one is created)
-- 2 = Author Last Name - Title
-- 3 = Title - Author Last Name
set nameFormat to 3
--Set the divider between the name and author in the group name and note file name:
set nameDivider to " - "
--Do you wish to enable logging?
set enableLog to true
--Set the name of the file in which all the changes will be logged and the title on the first line of the logfile:
set logFile to "Zotero Log"
set logTitle to "Zotero to DEVONthink Log" & return & return & return
--END CONFIGURATION
---------------------------------
--Thanks to Julio @ http://macscripter.net/viewtopic.php?id=24473 for this:
on split(someText, delimiter)
set AppleScript's text item delimiters to delimiter
set someText to someText's text items
set AppleScript's text item delimiters to {""} --> restore delimiters to default value
return someText
end split
--Deletes an item of an array:
on delItem(sourceList, itemContent)
set newList to {}
repeat with myCount from 1 to (count sourceList)
set myVar to (sourceList's item myCount) as string
if id of myVar is not id of itemContent then
--display dialog ("'" & myVar & "' IS '" & itemContent & "'")
set newList's end to sourceList's item myCount
end if
end repeat
--display dialog ((newList as string) & " " & itemContent)
return newList
end delItem
--Looks up the location of a group
on lookupLocation(collectionArray, collectionWanted)
set myLocation to ""
log item 2 of item 1 of collectionArray
repeat with m in collectionArray
--display dialog (item 1 of m as string)
if id of (item 1 of (item 1 of m) as string) is id of collectionWanted then
--display dialog (location of (item 2 of m) as string)
set myLocation to (item 2 of m) -- ((location of (item 2 of m) as string) & (item 1 of (item 1 of m)))
end if
end repeat
return myLocation
end lookupLocation
--thanks to deweller @ http://codesnippets.joyent.com/posts/show/1124 for this:
on appIsRunning(appname)
tell application "System Events" to (name of processes) contains appname
end appIsRunning
set restartFire to false
--Logs a variable list of log entries to the log file:
on LogEntries(LogText)
--assemble a string for the log based on the list of logged items:
set newText to ""
repeat with myitem in LogText
set newText to newText & myitem & return
end repeat
set LogText to newText
tell application id "com.devon-technologies.thinkpro2"
activate
try
set theDatabase to current database
--Create the log file if it does not exist:
if not (exists record at groupName & logFile) then
--Create the groupName folder if it doesn't exist (probably already done elsewhere in the script)
set logFolder to create location groupName in theDatabase
set newlogFile to create record with {name:logFile, type:txt, rich text:logTitle} in logFolder
end if
--Get the log file:
set logHandler to get record at groupName & logFile in theDatabase
--put the date of the log on an initial line:
set this_date to (the year of the (current date) as string) & "." & (the month of the (current date) as number as string) & "." & (the day of the (current date) as string) & " " & the time string of the (current date)
set the LogText to this_date & return & "-------------------------" & return & LogText
set the plain text of logHandler to the plain text of logHandler & return & LogText & return
on error error_message number error_number
if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
end try
end tell
end LogEntries
--Set up an empty log for adding log entries to:
set myLog to {}
-- ----------------------------------------
--HANDLE QUIT FOR FIREFOX IF IT IS RUNNING:
if appIsRunning("firefox-bin") then
if firefoxWarn is true then
set theReturnedItems to (display dialog "DEVONthink cannot read from the Zotero database while Firefox is running. Do you wish to quit Firefox and continue running the script?" buttons {"Cancel", "Quit Firefox"} default button 2)
set myAnswer to button returned of theReturnedItems
else
set myAnswer to "Quit Firefox"
end if
if myAnswer is "Quit Firefox" then
tell application "Firefox"
quit
end tell
delay 2
--Firefox won't quit immediately so delay up to 15 seconds before proceeding.
set myCount to 0
repeat while appIsRunning("firefox-bin")
delay 1
set myCount to myCount + 1
if myCount > 15 then
display dialog "The script was not able to quit Firefox."
exit repeat
end if
end repeat
set restartFire to true
end if
end if
--Thanks to Adam Bell @ http://macscripter.net/viewtopic.php?id=24744 for help with sqlite3 commands:
set loc to space & zoteroDatabase & space
set head to "sqlite3" & loc & quote
set tail to quote
--Get the names of the collections
set collectionData to "select collectionName,collectionID,parentCollectionID from collections;"
set collectionDataString to do shell script head & collectionData & tail
--get each collection name in an array
set collectionDataList to split(collectionDataString, "
")
--Get just the titles from all non-attachment items in the Zotero database
set titlesQuery to "select value, itemData.itemID from itemDataValues, itemData, items where itemData.valueID=itemDataValues.valueID and items.itemID=itemData.itemID and fieldID='110' and itemTypeID!='14';"
set titlesString to do shell script head & titlesQuery & tail
--get each title and itemID
set titlesList to split(titlesString, "
")
tell application id "com.devon-technologies.thinkpro2"
activate
try
--BEGIN CREATION OF THE COLLECTION HIERARCHY:
-- ------------------------------------------
--create a new group for each collection:
--duplicate the collection list to delete items for
set deathCamp to collectionDataList
--don't let it repeat for ever if something goes wrong:
set timeOutCount to 0
set createdCollections to {}
--repeat while there are collection groups left to create:
repeat while (count deathCamp) is not 0
repeat with i in deathCamp
--get the name, id, and parent of the collection:
set collectionInfo to split(i, "|") of me
--if it has no parent, create the group, then add it to the createdCollections list
if item 3 of collectionInfo is "" then
if not (exists record at groupName & (item 1 of collectionInfo)) then
--log this:
set the end of myLog to "CREATED group: " & groupName & (item 1 of collectionInfo)
end if
set newCollection to create location groupName & (item 1 of collectionInfo)
set the end of createdCollections to {collectionInfo, newCollection}
set deathCamp to delItem(deathCamp, i) of me
else --it has a parent
--check to see if the parent has been added:
repeat with x in createdCollections
if item 3 of collectionInfo is (item 2 of item 1 of x) then
if not (exists record at ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo) then
--log this:
set the end of myLog to "CREATED group: " & ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo
end if
--the parent has been found, create a new group under that location:
set newCollection to create location ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo
set the end of createdCollections to {collectionInfo, newCollection}
set deathCamp to delItem(deathCamp, i) of me
end if
end repeat
--if the parent hasn't been added yet (can't be found), then continue repeating through until it has been added
end if
end repeat
set timeOutCount to timeOutCount + 1
if timeOutCount is greater than 200 then
exit repeat
end if
end repeat
--END CREATION OF COLLECTION HIERARCHY
-- ------------------------------------
-- ------------------------------------
--BEGIN CREATION OF GROUP FOR SOURCE AND NOTE FILE FOR SOURCE
set myCount to 0
repeat with titleItem in titlesList
--split the title and the itemID
set titleItemSplit to split(titleItem, "|") of me
--Get the title of the work which is item 1 (item 2 is the id of the item)
set sourceName to item 1 of titleItemSplit
--Get the last name of the first author of type author/contributor/editor, if there is one
--IMPROVE THIS SECTION by moving this outside of the loop:
set authorQuery to "select lastName from creatorData, itemCreators where itemCreators.creatorID=creatorData.creatorDataID and itemCreators.itemID='" & item 2 of titleItemSplit & "' and creatorTypeID<'4';"
set authorString to do shell script head & authorQuery & tail
set authorList to {}
set authorList to split(authorString, "
") of me
--If there is one or more authors, take the first one and format the name as per nameFormat
if authorList is not {} and item 1 of authorList is not "" then
--If nameFormat is 1 (just the title) then leave the sourceName alone
if nameFormat is 2 then
--set it to author - title
set sourceName to item 1 of authorList & nameDivider & sourceName
else
set sourceName to sourceName & nameDivider & item 1 of authorList
end if
end if
set createTime to current date
set theDatabase to current database
--check to see if there is already a group for the source, if not then add it
if not (exists record at groupName & allName & "/" & sourceName) then
--get the categories and tags of the source and put them together if collectionTags is true
--IMPROVE THIS SECTION by moving this outside the loop:
set sourceID to item 2 of titleItemSplit
set collectionsQuery to "select collectionName from collections, collectionItems where collectionItems.collectionID=collections.collectionID and itemID='" & sourceID & "'"
set collectionsString to do shell script head & collectionsQuery & tail
set collectionsList to split(collectionsString, "
") of me
--get the tags of the source
--IMPROVE THIS SECTION by moving this out of the loop:
set tagsQuery to "select name from tags, itemTags where itemTags.tagID=tags.tagID and itemID='" & sourceID & "'"
set tagsString to do shell script head & tagsQuery & tail
set tagsList to split(tagsString, "
") of me
--CREATE THE GROUP for the source and the main note file within it.
set theGroup to create location groupName & allName & "/" & sourceName & "/" in theDatabase
--log this:
set the end of myLog to "CREATED group: " & groupName & allName & "/" & sourceName & "/"
--Also assign these tags to the source group as well
set the tags of theGroup to ((the tags of theGroup) & tagsList)
--CREATE THE NOTE FILE within the group for the source:
set newRecord to create record with {name:sourceName, type:rtf, rich text:sourceName, tags:tagsList} in theGroup
--log this:
set the end of myLog to "CREATED file: " & groupName & allName & "/" & sourceName & "/" & sourceName
--REPLICATE THE GROUP FOLDER TO COLLECTION:
repeat with targetCollection in collectionsList
set targetLocation to (location of lookupLocation(createdCollections, targetCollection) of me & targetCollection & "/")
if not (exists record at (targetLocation & sourceName)) then
replicate record theGroup to get record at (targetLocation)
--log this:
set the end of myLog to "REPLICATED group to: " & targetLocation & sourceName
end if
end repeat
set myCount to myCount + 1
end if
end repeat
if myCount > 0 then
display dialog (myCount as text) & " new sources were added to DEVONthink from your Zotero database."
else
display dialog "No new sources were added to DEVONthink database."
end if
on error error_message number error_number
if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
end try
end tell
if enableLog is true then
LogEntries(myLog)
end if
if restartFire is true then
tell application "/Applications/Firefox.app"
launch
end tell
end if