DEVONtech and Zotero

I created an applescript to import all my Zotero entries, create a folder for each source, add a main note file for the source, and assign it tags for all the Zotero tags and collections in the original source. It will only add them to the DEVONthink Pro database only if they have not already been added before.

You can read some of my notes on this and download the script from here:

muninn.net/blog/2010/09/zotero-a … think.html

I welcome criticism and improvements on the script. In the posting above I listed some of the ways I think can be improved.

I think there are a lot of possibilities for stronger integration between an open source bibliographic tool like Zotero and the powerful DEVONthink software and I hope better scripts will emerge in the future since I am not really skilled enough to get it do everything I’d like it to do.

The script source is below:


--CONFIGURATION - You need to set a few variables to get this to work for you, then put it in the DEVONthink script folder:

--Change the group name to /[group name for your notes]/
set groupName to "/Test/"

--Change the location below to the exact location of your zotero.sqlite. You can find it in Zotero advanced preferences "Show Data Directory"
set zoteroDatabase to "'~/Library/Application Support/Firefox/Profiles/[profile]/zotero/zotero.sqlite'"

--END CONFIGURATION

---------------------------------


--Thanks to Julio @ macscripter.net for this:
on split(someText, delimiter)
	set AppleScript's text item delimiters to delimiter
	set someText to someText's text items
	set AppleScript's text item delimiters to {""} --> restore delimiters to default value
	return someText
end split


--Thanks to Adam Bell @ macscripter.net for help with sqlite3 commands:
set loc to space & zoteroDatabase & space
set head to "sqlite3" & loc & quote
set tail to quote

--Get just the titles from all non-attachment items in the Zotero database
set titlesQuery to "select value, itemData.itemID from itemDataValues, itemData, items where itemData.valueID=itemDataValues.valueID and items.itemID=itemData.itemID and fieldID='110' and itemTypeID!='14';"
display dialog head & titlesQuery & tail
set titlesString to do shell script head & titlesQuery & tail

--get each title and itemID
set titlesList to split(titlesString, "
")

--display dialog "Got the titles. " & the number of items in titlesList

tell application id "com.devon-technologies.thinkpro2"
	activate
	try
		set myCount to 0
		
		repeat with titleItem in titlesList
			
			--split the title and the itemID
			--display dialog "titleItem: " & titleItem
			set titleItemSplit to split(titleItem, "|") of me
			--display dialog titleItemSplit
			set sourceName to item 1 of titleItemSplit
			
			set createTime to current date
			set theDatabase to current database
			if not (exists record at groupName & sourceName) then
				
				--get the categories and tags of the source and put them together: they will all be treated as tags
				set sourceID to item 2 of titleItemSplit
				
				--get the categories of the source
				set collectionsQuery to "select collectionName from collections, collectionItems where collectionItems.collectionID=collections.collectionID and itemID='" & sourceID & "'"
				set collectionsString to do shell script head & collectionsQuery & tail
				set collectionsList to split(collectionsString, "
") of me
				
				--get the tags of the source
				set tagsQuery to "select name from tags, itemTags where itemTags.tagID=tags.tagID and itemID='" & sourceID & "'"
				set tagsString to do shell script head & tagsQuery & tail
				set tagsList to split(collectionsString, "
") of me
				
				--merge them
				set tagsList to collectionsList & tagsList
				
				--Create the group for the source and the main note file within it.
				set theGroup to create location groupName & sourceName in theDatabase
				set newRecord to create record with {name:sourceName, type:rtf, rich text:sourceName, tags:tagsList} in theGroup
				
				set myCount to myCount + 1
			end if
			
		end repeat
		
		if myCount > 0 then
			display dialog (myCount as text) & " new sources were added to DEVONthink from your Zotero database."
		else
			display dialog "No new sources were added to DEVONthink database."
		end if
		
		
	on error error_message number error_number
		if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
	end try
end tell

@kmlawson

This is a very clever and useful script. Thank you for the time and effort that went into it. You’ve come up with a valuable addition to DT’s integration with our other apps.

Thanks for the kind words. Hope someone finds ways to further improve it.

Ah, there is a “display dialog” message or two in there that can be deleted from the script as displayed in the posting above. I don’t think it is in the downloadable version from my website.

Also, I noticed that if Firefox is running, the database is locked and the script won’t run. I have updated the script on my site but esssentially all I did was add the following under the split function:

--thanks to deweller @ http://codesnippets.joyent.com/posts/show/1124 for this:
on appIsRunning(appName)
    tell application "System Events" to (name of processes) contains appName
end appIsRunning

if appIsRunning("firefox-bin") then
    set theReturnedItems to (display dialog "DEVONthink cannot read from the Zotero database while Firefox is running. Do you wish to quit Firefox and continue running the script?" buttons {"Cancel", "Quit Firefox"} default button 2)
    if the button returned of theReturnedItems is "Quit Firefox" then
        tell application "Firefox"
            quit
        end tell
        delay 1
        
        --Firefox won't quit immediately so delay up to 15 seconds before proceeding.
        set myCount to 0
        repeat while appIsRunning("firefox-bin")
            delay 1
            set myCount to myCount + 1
            if myCount > 15 then
                display dialog "The script was not able to quit Firefox."
                exit repeat
            end if
        end repeat
        
    end if
end if

I have made major changes/improvements to the Zotero import script. Full script is below but you can read more and download it from the home page I mentioned in my first posting. The features added are found in the comments at the beginning of the script. Thanks to various people for their help in learning how DEVONthink interacts with applescript.

--Zotero to DEVONThink Pro 1.5

(*

by K. M. Lawson 
http://muninn.net/

Please let me know if you find ways to improve on this script and share it with others. Feel free to edit or delete the comments below as needed:

This script was designed for my own uses. I have the following setup and need:

-I often scrape useful citation information from web databases into the Firefox Zotero plugin
-In DEVONThink Pro, in a folder/group called "Sources" I keep a folder for each source
-In each source folder I create a rich text file with some general notes on the source, timeline, and key concepts. I may also keep a PDF of an article and sometimes I have a timeline here in this folder as well.
-When taking notes on the source, I use a "Note on Source" template for DEVONThink of my own creation which makes a new note file in this folder, using the same tags as the main note file. Each fragment or cluster of related notes gets its own file which can then be grouped and organized later as if they were separate note cards.
-I want to automate the process of creating folders and a main note file for every Zotero bibliographic entry. 

This script does the following:

-Whenever the script is run, it will get a list of all titles in the Zotero database that aren't attachments
-It will see if a group with the name of the source already exists in DEVONthink Pro. If it does, it moves on to the next entry.
-If there is no such group, one is created
-In this new group a new rich text file is created to be the main note file for that source
-The new rich text file in the group for the source is given tags corresponding to whatever collections and tags it had in Zotero

Notes and Known Issues:

-Please note that if you have a large Zotero database, this script can take a long time to run. You may need to wrap this in "with timeout" clause. I haven't tested this with very large databases.

-Please note: While you can update your list based on new additions to the Zotero database when you like, keep in mind that if you rename the groups in DEVONthink such that they are no long identical to the title in Zotero, this script will assume you do not have an entry for that source and create a new one.

-Although this script only reads data from your Zotero database and thus should do it no harm, it is always good to keep good backups of Zotero files. Use the script at your own risk.

-If you move or change the hierarchy of folders in Zotero and run the script again later, it will not delete the old groups or move them (of course, neither will it recreate folders in old locations, so after running simply go through and move and replace the newly created groups (if you have notes in them, just delete them in the old location if you don't)

WAYS TO IMPROVE THIS SCRIPT:

-make a few big SQL calls at the beginning and store all the title, author, and collection data at once instead of making SQL calls to the Zotero database during every loop - will reduce the number of calls and speed up the script.

UPDATED 2010.9.10 1.1

-Makes a copy of the Zotero database and puts it in the tmp file, then deletes this copy when done. Otherwise, an open Firefox will prevent the reading of the file.

UPDATED 2010.9.15 1.5

-Now puts all sources into an "_All" folder (or whatever you call it) 
-Puts a replicate of each source into a collection group corresponding to that in Zotero
-Option to not warn about Firefox
-Options to set the formatting of the name of the group and note file and its divider


*)

--CONFIGURATION - You need to set a few variables to get this to work for you, then put it in the DEVONthink script folder:

--Change the group name to /[group name for your notes]/
set groupName to "/Sources/"

--Do you wish the collections to also be assigned as regular tags to the items in addition to created as groups? If so, set this to true
set collectionTags to false

--If you don't want the folder with all the sources to be called _All change it here:
set allName to "_All"

--If you want to be warned before Firefox is quit, set this to true:
set firefoxWarn to false

--Change the location below to the exact location of your zotero.sqlite. You can find it in Zotero advanced preferences "Show Data Directory"
set zoteroDatabase to "'~/Library/Application Support/Firefox/Profiles/[profile]/zotero/zotero.sqlite'"

--Set the type of group name you want
-- 1 = just the title of the work (Note: if you have multiple works with same title, only one is created)
-- 2 = Author Last Name - Title
-- 3 = Title - Author Last Name
set nameFormat to 3

--Set the divider between the name and author in the group name and note file name:
set nameDivider to " - "

--END CONFIGURATION

---------------------------------


--Thanks to Julio @ http://macscripter.net/viewtopic.php?id=24473 for this:
on split(someText, delimiter)
    set AppleScript's text item delimiters to delimiter
    set someText to someText's text items
    set AppleScript's text item delimiters to {""} --> restore delimiters to default value
    return someText
end split

--Deletes an item of an array:
on delItem(sourceList, itemContent)
    set newList to {}
    
    repeat with myCount from 1 to (count sourceList)
        set myVar to (sourceList's item myCount) as string
        if id of myVar is not id of itemContent then
            --display dialog ("'" & myVar & "' IS '" & itemContent & "'")
            set newList's end to sourceList's item myCount
        end if
    end repeat
    --display dialog ((newList as string) & " " & itemContent)
    return newList
end delItem

--Looks up the location of a group
on lookupLocation(collectionArray, collectionWanted)
    set myLocation to ""
    log item 2 of item 1 of collectionArray
    repeat with m in collectionArray
        --display dialog (item 1 of m as string)
        if id of (item 1 of (item 1 of m) as string) is id of collectionWanted then
            --display dialog (location of (item 2 of m) as string)
            set myLocation to (item 2 of m) -- ((location of (item 2 of m) as string) & (item 1 of (item 1 of m))) 
        end if
    end repeat
    return myLocation
end lookupLocation

--thanks to deweller @ http://codesnippets.joyent.com/posts/show/1124 for this:
on appIsRunning(appname)
    tell application "System Events" to (name of processes) contains appname
end appIsRunning

set restartFire to false


-- ----------------------------------------
--HANDLE QUIT FOR FIREFOX IF IT IS RUNNING:

if appIsRunning("firefox-bin") then
    
    if firefoxWarn is true then
        set theReturnedItems to (display dialog "DEVONthink cannot read from the Zotero database while Firefox is running. Do you wish to quit Firefox and continue running the script?" buttons {"Cancel", "Quit Firefox"} default button 2)
        set myAnswer to button returned of theReturnedItems
    else
        set myAnswer to "Quit Firefox"
    end if
    
    if myAnswer is "Quit Firefox" then
        
        tell application "Firefox"
            quit
            
        end tell
        delay 2
        
        --Firefox won't quit immediately so delay up to 15 seconds before proceeding.
        set myCount to 0
        repeat while appIsRunning("firefox-bin")
            delay 1
            set myCount to myCount + 1
            if myCount > 15 then
                display dialog "The script was not able to quit Firefox."
                exit repeat
            end if
        end repeat
        set restartFire to true
    end if
end if




--Thanks to Adam Bell @ http://macscripter.net/viewtopic.php?id=24744 for help with sqlite3 commands:
set loc to space & zoteroDatabase & space
set head to "sqlite3" & loc & quote
set tail to quote

--Get the names of the collections
set collectionData to "select collectionName,collectionID,parentCollectionID from collections;"
set collectionDataString to do shell script head & collectionData & tail

--get each collection name in an array
set collectionDataList to split(collectionDataString, "
")

--Get just the titles from all non-attachment items in the Zotero database
set titlesQuery to "select value, itemData.itemID from itemDataValues, itemData, items where itemData.valueID=itemDataValues.valueID and items.itemID=itemData.itemID and fieldID='110' and itemTypeID!='14';"
set titlesString to do shell script head & titlesQuery & tail

--get each title and itemID
set titlesList to split(titlesString, "
")

tell application id "com.devon-technologies.thinkpro2"
    activate
    try
        
        
        --BEGIN CREATION OF THE COLLECTION HIERARCHY:
        -- ------------------------------------------
        
        --create a new group for each collection:
        
        --duplicate the collection list to delete items for
        set deathCamp to collectionDataList
        
        --don't let it repeat for ever if something goes wrong:
        set timeOutCount to 0
        
        set createdCollections to {}
        
        --repeat while there are collection groups left to create:
        repeat while (count deathCamp) is not 0
            
            
            
            repeat with i in deathCamp
                --get the name, id, and parent of the collection:
                set collectionInfo to split(i, "|") of me
                
                --if it has no parent, create the group, then add it to the createdCollections list
                if item 3 of collectionInfo is "" then
                    set newCollection to create location groupName & (item 1 of collectionInfo)
                    set the end of createdCollections to {collectionInfo, newCollection}
                    set deathCamp to delItem(deathCamp, i) of me
                else --it has a parent
                    --check to see if the parent has been added:
                    repeat with x in createdCollections
                        if item 3 of collectionInfo is (item 2 of item 1 of x) then
                            --the parent has been found, create a new group under that location:
                            set newCollection to create location ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo
                            set the end of createdCollections to {collectionInfo, newCollection}
                            set deathCamp to delItem(deathCamp, i) of me
                        end if
                    end repeat
                    --if the parent hasn't been added yet (can't be found), then continue repeating through until it has been added
                end if
            end repeat
            set timeOutCount to timeOutCount + 1
            if timeOutCount is greater than 200 then
                exit repeat
            end if
        end repeat
        
        
        --END CREATION OF COLLECTION HIERARCHY        
        -- ------------------------------------
        
        
        -- ------------------------------------
        --BEGIN CREATION OF GROUP FOR SOURCE AND NOTE FILE FOR SOURCE
        
        set myCount to 0
        
        repeat with titleItem in titlesList
            
            --split the title and the itemID
            set titleItemSplit to split(titleItem, "|") of me
            
            --Get the title of the work which is item 1 (item 2 is the id of the item)
            set sourceName to item 1 of titleItemSplit
            
            --Get the last name of the first author of type author/contributor/editor, if there is one
            --IMPROVE THIS SECTION by moving this outside of the loop:
            set authorQuery to "select lastName from creatorData, itemCreators where itemCreators.creatorID=creatorData.creatorDataID and itemCreators.itemID='" & item 2 of titleItemSplit & "' and creatorTypeID<'4';"
            set authorString to do shell script head & authorQuery & tail
            set authorList to {}
            set authorList to split(authorString, "
") of me
            --If there is one or more authors, take the first one and format the name as per nameFormat
            if authorList is not {} and item 1 of authorList is not "" then
                --If nameFormat is 1 (just the title) then leave the sourceName alone
                if nameFormat is 2 then
                    --set it to author - title
                    set sourceName to item 1 of authorList & nameDivider & sourceName
                else
                    set sourceName to sourceName & nameDivider & item 1 of authorList
                end if
            end if
            
            
            set createTime to current date
            set theDatabase to current database
            
            --check to see if there is already a group for the source, if not then add it
            if not (exists record at groupName & allName & "/" & sourceName) then
                
                --get the categories and tags of the source and put them together if collectionTags is true    
                --IMPROVE THIS SECTION by moving this outside the loop:            
                set sourceID to item 2 of titleItemSplit
                set collectionsQuery to "select collectionName from collections, collectionItems where collectionItems.collectionID=collections.collectionID and itemID='" & sourceID & "'"
                set collectionsString to do shell script head & collectionsQuery & tail
                set collectionsList to split(collectionsString, "
") of me
                
                
                --get the tags of the source
                --IMPROVE THIS SECTION by moving this out of the loop:
                set tagsQuery to "select name from tags, itemTags where itemTags.tagID=tags.tagID and itemID='" & sourceID & "'"
                set tagsString to do shell script head & tagsQuery & tail
                set tagsList to split(tagsString, "
") of me
                
                --CREATE THE GROUP for the source and the main note file within it.
                set theGroup to create location groupName & allName & "/" & sourceName & "/" in theDatabase
                
                --Also assign these tags to the source group as well
                set the tags of theGroup to ((the tags of theGroup) & tagsList)
                
                --CREATE THE NOTE FILE within the group for the source:
                set newRecord to create record with {name:sourceName, type:rtf, rich text:sourceName, tags:tagsList} in theGroup
                
                
                
                --REPLICATE THE GROUP FOLDER TO COLLECTION:
                repeat with targetCollection in collectionsList
                    
                    set targetLocation to (location of lookupLocation(createdCollections, targetCollection) of me & targetCollection & "/")
                    
                    if not (exists record at (targetLocation & sourceName)) then
                        replicate record theGroup to get record at (targetLocation)
                    end if
                end repeat
                
                set myCount to myCount + 1
            end if
            
        end repeat
        
        if myCount > 0 then
            display dialog (myCount as text) & " new sources were added to DEVONthink from your Zotero database."
        else
            display dialog "No new sources were added to DEVONthink database."
        end if
        
    on error error_message number error_number
        if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
    end try
end tell

if restartFire is true then
    tell application "/Applications/Firefox.app"
        launch
    end tell
end if
A

Hi KMLawson,
Thanks for this great script.

Is it compatible with DTP 2.03 ?
I see in the top of the script it says “Zotero to DEVONThink Pro 1.5” and I am not sure if this is referring to version 1.5 of your script or version 1.5 of DT.

Cheers,

Jon

Hello Jon,

Thanks! I wrote the script and tested on DEVONthink 2.0.5, the 1.5 was only the version of my script.

I have made a few changes to the script to support logging. That is, instead of simply telling you how many sources were added, it will log the exact changes made to a plain text file so you can see exactly what groups are created, note files created, and where they are replicated to:

--Zotero to DEVONThink Pro 1.6

(*

by K. M. Lawson 
http://muninn.net/

Please let me know if you find ways to improve on this script and share it with others. Feel free to edit or delete the comments below as needed:

This script was designed for my own uses. I have the following setup and need:

-I often scrape useful citation information from web databases into the Firefox Zotero plugin
-In DEVONThink Pro, in a folder/group called "Sources" I keep a folder for each source
-In each source folder I create a rich text file with some general notes on the source, timeline, and key concepts. I may also keep a PDF of an article and sometimes I have a timeline here in this folder as well.
-When taking notes on the source, I use a "Note on Source" template for DEVONThink of my own creation which makes a new note file in this folder, using the same tags as the main note file. Each fragment or cluster of related notes gets its own file which can then be grouped and organized later as if they were separate note cards.
-I want to automate the process of creating folders and a main note file for every Zotero bibliographic entry. 

This script does the following:

-Whenever the script is run, it will get a list of all titles in the Zotero database that aren't attachments
-It will see if a group with the name of the source already exists in DEVONthink Pro. If it does, it moves on to the next entry.
-If there is no such group, one is created
-In this new group a new rich text file is created to be the main note file for that source
-The new rich text file in the group for the source is given tags corresponding to whatever collections and tags it had in Zotero

Notes and Known Issues:

-Please note that if you have a large Zotero database, this script can take a long time to run. You may need to wrap this in "with timeout" clause. I haven't tested this with very large databases.

-Please note: While you can update your list based on new additions to the Zotero database when you like, keep in mind that if you rename the groups in DEVONthink such that they are no long identical to the title in Zotero, this script will assume you do not have an entry for that source and create a new one.

-If you have a Zotero title that is also exactly the same as a collection name or tag, for every source with the collection/tag, you will end up with replicate note files in both its own folder and in the folder with the same name as the tag/collection

-Although this script only reads data from your Zotero database and thus should do it no harm, it is always good to keep good backups of Zotero files. Use the script at your own risk.

-If you move or change the hierarchy of folders in Zotero and run the script again later, it will not delete the old groups or move them (of course, neither will it recreate folders in old locations, so after running simply go through and move and replace the newly created groups (if you have notes in them, just delete them in the old location if you don't)

WAYS TO IMPROVE THIS SCRIPT:

-make a few big SQL calls at the beginning and store all the title, author, and collection data at once instead of making SQL calls to the Zotero database during every loop - will reduce the number of calls and speed up the script.

UPDATED 2010.9.10 1.1

-Makes a copy of the Zotero database and puts it in the tmp file, then deletes this copy when done. Otherwise, an open Firefox will prevent the reading of the file.

UPDATED 2010.9.15 1.5

-Now puts all sources into an "_All" folder (or whatever you call it) 
-Puts a replicate of each source into a collection group corresponding to that in Zotero
-Option to not warn about Firefox
-Options to set the formatting of the name of the group and note file and its divider

UPDATED 2010.10.16

-added ability to log all changes made

*)

global groupName
global logFile
global logTitle

--CONFIGURATION - You need to set a few variables to get this to work for you, then put it in the DEVONthink script folder:

--Change the group name to /[group name for your notes]/
set groupName to "/Sources/"

--Do you wish the collections to also be assigned as regular tags to the items in addition to created as groups? If so, set this to true
set collectionTags to false

--If you don't want the folder with all the sources to be called _All change it here:
set allName to "_All"

--If you want to be warned before Firefox is quit, set this to true:
set firefoxWarn to false

--Change the location below to the exact location of your zotero.sqlite. You can find it in Zotero advanced preferences "Show Data Directory"
set zoteroDatabase to "'~/Library/Application Support/Firefox/Profiles/[profile]/zotero/zotero.sqlite'"

--Set the type of group name you want
-- 1 = just the title of the work (Note: if you have multiple works with same title, only one is created)
-- 2 = Author Last Name - Title
-- 3 = Title - Author Last Name
set nameFormat to 3

--Set the divider between the name and author in the group name and note file name:
set nameDivider to " - "

--Do you wish to enable logging?
set enableLog to true

--Set the name of the file in which all the changes will be logged and the title on the first line of the logfile:
set logFile to "Zotero Log"
set logTitle to "Zotero to DEVONthink Log" & return & return & return




--END CONFIGURATION

---------------------------------


--Thanks to Julio @ http://macscripter.net/viewtopic.php?id=24473 for this:
on split(someText, delimiter)
    set AppleScript's text item delimiters to delimiter
    set someText to someText's text items
    set AppleScript's text item delimiters to {""} --> restore delimiters to default value
    return someText
end split

--Deletes an item of an array:
on delItem(sourceList, itemContent)
    set newList to {}
    
    repeat with myCount from 1 to (count sourceList)
        set myVar to (sourceList's item myCount) as string
        if id of myVar is not id of itemContent then
            --display dialog ("'" & myVar & "' IS '" & itemContent & "'")
            set newList's end to sourceList's item myCount
        end if
    end repeat
    --display dialog ((newList as string) & " " & itemContent)
    return newList
end delItem

--Looks up the location of a group
on lookupLocation(collectionArray, collectionWanted)
    set myLocation to ""
    log item 2 of item 1 of collectionArray
    repeat with m in collectionArray
        --display dialog (item 1 of m as string)
        if id of (item 1 of (item 1 of m) as string) is id of collectionWanted then
            --display dialog (location of (item 2 of m) as string)
            set myLocation to (item 2 of m) -- ((location of (item 2 of m) as string) & (item 1 of (item 1 of m))) 
        end if
    end repeat
    return myLocation
end lookupLocation

--thanks to deweller @ http://codesnippets.joyent.com/posts/show/1124 for this:
on appIsRunning(appname)
    tell application "System Events" to (name of processes) contains appname
end appIsRunning

set restartFire to false

--Logs a variable list of log entries to the log file:
on LogEntries(LogText)
    
    --assemble a string for the log based on the list of logged items:
    set newText to ""
    repeat with myitem in LogText
        set newText to newText & myitem & return
    end repeat
    set LogText to newText
    
    tell application id "com.devon-technologies.thinkpro2"
        activate
        try
            set theDatabase to current database
            
            --Create the log file if it does not exist:
            if not (exists record at groupName & logFile) then
                
                --Create the groupName folder if it doesn't exist (probably already done elsewhere in the script)
                set logFolder to create location groupName in theDatabase
                set newlogFile to create record with {name:logFile, type:txt, rich text:logTitle} in logFolder
                
            end if
            
            --Get the log file:
            set logHandler to get record at groupName & logFile in theDatabase
            
            --put the date of the log on an initial line:
            set this_date to (the year of the (current date) as string) & "." & (the month of the (current date) as number as string) & "." & (the day of the (current date) as string) & " " & the time string of the (current date)
            set the LogText to this_date & return & "-------------------------" & return & LogText
            
            set the plain text of logHandler to the plain text of logHandler & return & LogText & return
            
            
            
        on error error_message number error_number
            if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
        end try
    end tell
end LogEntries
--Set up an empty log for adding log entries to:
set myLog to {}


-- ----------------------------------------
--HANDLE QUIT FOR FIREFOX IF IT IS RUNNING:

if appIsRunning("firefox-bin") then
    
    if firefoxWarn is true then
        set theReturnedItems to (display dialog "DEVONthink cannot read from the Zotero database while Firefox is running. Do you wish to quit Firefox and continue running the script?" buttons {"Cancel", "Quit Firefox"} default button 2)
        set myAnswer to button returned of theReturnedItems
    else
        set myAnswer to "Quit Firefox"
    end if
    
    if myAnswer is "Quit Firefox" then
        
        tell application "Firefox"
            quit
            
        end tell
        delay 2
        
        --Firefox won't quit immediately so delay up to 15 seconds before proceeding.
        set myCount to 0
        repeat while appIsRunning("firefox-bin")
            delay 1
            set myCount to myCount + 1
            if myCount > 15 then
                display dialog "The script was not able to quit Firefox."
                exit repeat
            end if
        end repeat
        set restartFire to true
    end if
end if




--Thanks to Adam Bell @ http://macscripter.net/viewtopic.php?id=24744 for help with sqlite3 commands:
set loc to space & zoteroDatabase & space
set head to "sqlite3" & loc & quote
set tail to quote

--Get the names of the collections
set collectionData to "select collectionName,collectionID,parentCollectionID from collections;"
set collectionDataString to do shell script head & collectionData & tail

--get each collection name in an array
set collectionDataList to split(collectionDataString, "
")

--Get just the titles from all non-attachment items in the Zotero database
set titlesQuery to "select value, itemData.itemID from itemDataValues, itemData, items where itemData.valueID=itemDataValues.valueID and items.itemID=itemData.itemID and fieldID='110' and itemTypeID!='14';"
set titlesString to do shell script head & titlesQuery & tail

--get each title and itemID
set titlesList to split(titlesString, "
")

tell application id "com.devon-technologies.thinkpro2"
    activate
    try
        
        
        --BEGIN CREATION OF THE COLLECTION HIERARCHY:
        -- ------------------------------------------
        
        --create a new group for each collection:
        
        --duplicate the collection list to delete items for
        set deathCamp to collectionDataList
        
        --don't let it repeat for ever if something goes wrong:
        set timeOutCount to 0
        
        set createdCollections to {}
        
        --repeat while there are collection groups left to create:
        repeat while (count deathCamp) is not 0
            
            
            
            repeat with i in deathCamp
                --get the name, id, and parent of the collection:
                set collectionInfo to split(i, "|") of me
                
                --if it has no parent, create the group, then add it to the createdCollections list
                if item 3 of collectionInfo is "" then
                    if not (exists record at groupName & (item 1 of collectionInfo)) then
                        --log this:
                        set the end of myLog to "CREATED group: " & groupName & (item 1 of collectionInfo)
                    end if
                    set newCollection to create location groupName & (item 1 of collectionInfo)
                    set the end of createdCollections to {collectionInfo, newCollection}
                    set deathCamp to delItem(deathCamp, i) of me
                else --it has a parent
                    --check to see if the parent has been added:
                    repeat with x in createdCollections
                        if item 3 of collectionInfo is (item 2 of item 1 of x) then
                            if not (exists record at ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo) then
                                --log this:
                                set the end of myLog to "CREATED group: " & ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo
                            end if
                            
                            --the parent has been found, create a new group under that location:
                            set newCollection to create location ((location of item 2 of x) & (item 1 of item 1 of x)) & "/" & item 1 of collectionInfo
                            set the end of createdCollections to {collectionInfo, newCollection}
                            set deathCamp to delItem(deathCamp, i) of me
                        end if
                    end repeat
                    --if the parent hasn't been added yet (can't be found), then continue repeating through until it has been added
                end if
            end repeat
            set timeOutCount to timeOutCount + 1
            if timeOutCount is greater than 200 then
                exit repeat
            end if
        end repeat
        
        
        --END CREATION OF COLLECTION HIERARCHY        
        -- ------------------------------------
        
        
        -- ------------------------------------
        --BEGIN CREATION OF GROUP FOR SOURCE AND NOTE FILE FOR SOURCE
        
        set myCount to 0
        
        repeat with titleItem in titlesList
            
            --split the title and the itemID
            set titleItemSplit to split(titleItem, "|") of me
            
            --Get the title of the work which is item 1 (item 2 is the id of the item)
            set sourceName to item 1 of titleItemSplit
            
            --Get the last name of the first author of type author/contributor/editor, if there is one
            --IMPROVE THIS SECTION by moving this outside of the loop:
            set authorQuery to "select lastName from creatorData, itemCreators where itemCreators.creatorID=creatorData.creatorDataID and itemCreators.itemID='" & item 2 of titleItemSplit & "' and creatorTypeID<'4';"
            set authorString to do shell script head & authorQuery & tail
            set authorList to {}
            set authorList to split(authorString, "
") of me
            --If there is one or more authors, take the first one and format the name as per nameFormat
            if authorList is not {} and item 1 of authorList is not "" then
                --If nameFormat is 1 (just the title) then leave the sourceName alone
                if nameFormat is 2 then
                    --set it to author - title
                    set sourceName to item 1 of authorList & nameDivider & sourceName
                else
                    set sourceName to sourceName & nameDivider & item 1 of authorList
                end if
            end if
            
            
            set createTime to current date
            set theDatabase to current database
            
            --check to see if there is already a group for the source, if not then add it
            if not (exists record at groupName & allName & "/" & sourceName) then
                
                --get the categories and tags of the source and put them together if collectionTags is true    
                --IMPROVE THIS SECTION by moving this outside the loop:            
                set sourceID to item 2 of titleItemSplit
                set collectionsQuery to "select collectionName from collections, collectionItems where collectionItems.collectionID=collections.collectionID and itemID='" & sourceID & "'"
                set collectionsString to do shell script head & collectionsQuery & tail
                set collectionsList to split(collectionsString, "
") of me
                
                
                --get the tags of the source
                --IMPROVE THIS SECTION by moving this out of the loop:
                set tagsQuery to "select name from tags, itemTags where itemTags.tagID=tags.tagID and itemID='" & sourceID & "'"
                set tagsString to do shell script head & tagsQuery & tail
                set tagsList to split(tagsString, "
") of me
                
                --CREATE THE GROUP for the source and the main note file within it.
                set theGroup to create location groupName & allName & "/" & sourceName & "/" in theDatabase
                --log this:
                set the end of myLog to "CREATED group: " & groupName & allName & "/" & sourceName & "/"
                --Also assign these tags to the source group as well
                set the tags of theGroup to ((the tags of theGroup) & tagsList)
                
                --CREATE THE NOTE FILE within the group for the source:
                set newRecord to create record with {name:sourceName, type:rtf, rich text:sourceName, tags:tagsList} in theGroup
                --log this:
                set the end of myLog to "CREATED file: " & groupName & allName & "/" & sourceName & "/" & sourceName
                
                
                --REPLICATE THE GROUP FOLDER TO COLLECTION:
                repeat with targetCollection in collectionsList
                    
                    set targetLocation to (location of lookupLocation(createdCollections, targetCollection) of me & targetCollection & "/")
                    
                    if not (exists record at (targetLocation & sourceName)) then
                        replicate record theGroup to get record at (targetLocation)
                        --log this:
                        set the end of myLog to "REPLICATED group to: " & targetLocation & sourceName
                    end if
                end repeat
                
                set myCount to myCount + 1
            end if
            
        end repeat
        
        if myCount > 0 then
            display dialog (myCount as text) & " new sources were added to DEVONthink from your Zotero database."
        else
            display dialog "No new sources were added to DEVONthink database."
        end if
        
    on error error_message number error_number
        if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
    end try
end tell

if enableLog is true then
    LogEntries(myLog)
end if

if restartFire is true then
    tell application "/Applications/Firefox.app"
        launch
    end tell
end if

Although the script works for me and I haven’t personally faced these problems, I have received reports that the script gets hung up when there are certain characters in some titles of books etc. in Zotero.

It would be great if someone was willing to go through the script (text of the most recent version in the last posting on this thread) and see if they can fix it to handle all forms of unusual titles and perhaps otherwise improve on the work of a complete amateur such as myself.

Ideally, I think some super powerful and improved version of this script (which, pieced together as it is from lots of bits of code put on these helpful forums and elsewhere, I claim no rights over) or a completely new one which performs the same task, should be included in DEVONthink by default to help those of us who wish to use both of these powerful tools smoothly.

I completely agree! I’m impressed by this work, and I also agree with the dreams for future versions of the script. As a newbie, I’m wondering if someone could please tell me if it would be better to wait until this script is ready for prime time, or would it be best to load it and give it a try now? What happens if there are subsequent versions? Will I lose work? Will changes apply retroactively?

Since the script merely creates groups and files based on your Zotero database, it should not do anything destructive: it never deletes anything in DT.

I am still hoping someone might go through my script - since I am an amateur and perhaps help clean up the parts which might run into trouble with bad data or unusual characters in the Zotero database.

Again, I continue to use this script all the time - updating my DT with any new items I have added to Zotero recently. Still would love to have more professional eyes who can think more carefully about checking data integrity to help catch the problems some people are having with not all entries being imported.

Is it possible, using your script, KMLawson, to just export a group/folder from Zotero? So if I am working on project, have a group of say 50 sources, but don’t want my entire library, I can just export that?

I’ve tried in vain to get this script to work…after importing one or two items it stops, presumably because of unusual characters like colons. I took a look at the AppleScript, but I’m far too much of an amateur to be able to improve it.

Anybody else willing to take a look? It would be really great to be able to import Zotero into DEVONthink…

Any help would be much appreciated!

I would like to second the issue of the previous comment. The script works, but only for one book, one with a colon in it which I suspect is the problem. It would be great of someone could fix the issue, which I am betting is small. I am too unskilled to even know where to begin, however.

Dear K. M. Lawson,
Using DT I was missing just the two options - just the ones you took care of werer the ones I was missing. The Note taking option from your website works great.
But I cannot get the Apple Script running that should give me Zotero-structure in DT.
Even though I carefully added my zotero.sqlite folder in the configurations I keep getting that same error:

I am stuck, maybe someone could help me??

I’ve been trying to run the script, but keep getting the following error:

Any advice on what I should do? I’m very new to all of this, so any guidance would be greatly appreciated!

Also working with Zotero, and testing DTP wondering if I will buy it, I am very very much interested by KMLawson script. It runs on my mac, but… only generates one item (like adro and dtm222).
Where can be the bug? French accents in “Guide rapide pour débuter”?
It seems to only work through the first item entered in the sqlite when I first started Zotero, ignoring the real work I since registered in.
Any idea?

Removing the French “é”, it worked. But stopped on the second item, also containing accents…
Then again. But the next did not contained…
Is the problem really there?
It would be so nice…

Sounds like a unicode problem. I’ll email myself a reminder and see if I can’t fix up a working solution this evening… I might be able to replace some or all of the AppleScript with Python or Perl, which should make the script run a lot faster and work a lot more reliably.

EDIT: Or PHP, which would make the script get done a whole lot faster…

Will be great! Thanks a lot.

Sorry all… I haven’t had a chance to work on it yet. I’m being slaughtered by allergies. I’m feeling a bit less awful now, though, so I’ll be sure to work on it this weekend.