May I ask, what is the point of importing the data from The Brain?
Are you transitioning from it to DEVONthink?
Unlike DEVONthink, TheBrain is a locked proprietary platform. Though working with TheBrain is interesting and useful, getting off of it is chore. So, many people have collections of notes, references, and attachments in TheBrain that could reside just as well in DEVONthink. I think that’s a good enough motivation to continue the discussion and thank Pete31 for what he’s working on.
1 Like
This script should
- Import TheBrain’s JSON export …
- … structured like TheBrain’s Folders export …
- … create outgoing and incoming links in markdown records …
- … create links to attachments …
- … and optionally import external attachments.
It works reliably with my very small test data, I’m of course not sure whether it works as expected with real data.
Please test it even if you don’t want to use TheBrain’s export in DEVONthink, any feedback would be appreciated. I’m not satisfied with the look of the markdown records, however fine looking markdown doesn’t make sense if it doesn’t work …
Create a new folder and export your Brain as “JSON” and as “Folders” into two subfolders named “JSON” and “Folders”.
Hope it works 
Edit: There’s a new version, down in this thread.
-- Import TheBrain export
-- Create a new folder and export your Brain as "JSON" and as "Folders" into two subfolders named "JSON" and "Folders".
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
property useNameAsHeading : true
property theHeadingLevel : "###"
property theLinkSectionDelimiter : "---"
property theLinkDelimiter : " "
property importExternalAttachments : false
global thePlist
tell application "System Events"
try
activate
set theFolderPath to POSIX path of (choose folder with prompt "Choose folder that holds \"JSON\" and \"Folders\" subfolders:" default location (path to desktop folder) without invisibles)
set theExportAsFolderPath to (theFolderPath & "/" & "Folders") as string
my existsPath(theExportAsFolderPath)
if folders of folder theExportAsFolderPath = {} then error "Error: Missing \"Folders\" export"
set theExportAsJSONPath to (theFolderPath & "/" & "JSON") as string
my existsPath(theExportAsJSONPath)
set theExportAsJSONSubfolderPaths to POSIX path of folders of folder theExportAsJSONPath
if theExportAsJSONSubfolderPaths = {} then error "Error: Missing \"JSON\" export"
set theThoughtsJSON_Path to (theExportAsJSONPath & "/" & "thoughts.json") as string
my existsPath(theThoughtsJSON_Path)
set theLinksJSON_Path to (theExportAsJSONPath & "/" & "links.json") as string
my existsPath(theLinksJSON_Path)
set theAttachmentsJSON_Path to (theExportAsJSONPath & "/" & "attachments.json") as string
my existsPath(theAttachmentsJSON_Path)
set thePlist to (theFolderPath & "/" & "Brain.plist") as string
try
exists POSIX file thePlist as alias
on error
make new property list file with properties {name:thePlist}
end try
on error error_message number error_number
activate
if the error_number is not -128 then display alert "System Events" message error_message as warning
return
end try
end tell
tell application id "DNtp"
try
activate
set theGroup to display group selector
set theDatabase to database of theGroup
set theLocationAndName to ((location of theGroup) & (name of theGroup)) as string
show progress indicator "Importing Brain ..." steps 4
on error error_message number error_number
activate
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
return
end try
end tell
try
my stepProgress("Reading JSON")
set theJSON_LinksDictionary to my readJSON(theLinksJSON_Path)
repeat with thisItem in theJSON_LinksDictionary
set thisThoughtIdA to (|ThoughtIdA| of thisItem) as string
set thisThoughtIdB to (|ThoughtIdB| of thisItem) as string
my writePlist(thisThoughtIdA, missing value, thisThoughtIdB, missing value, missing value)
my writePlist(thisThoughtIdB, missing value, missing value, thisThoughtIdA, missing value)
end repeat
set theJSON_ThoughtsDictionary to my readJSON(theThoughtsJSON_Path)
repeat with thisItem in theJSON_ThoughtsDictionary
my writePlist((|Id| of thisItem) as string, (|Name| of thisItem) as string, missing value, missing value, missing value)
end repeat
set theJSON_AttachmentsDictionary to my readJSON(theAttachmentsJSON_Path)
repeat with thisItem in theJSON_AttachmentsDictionary
set thisName to (|Name| of thisItem) as string
set thisLocation to (|Location| of thisItem) as string
if thisName ≠ thisLocation then my writePlist((|SourceId| of thisItem) as string, missing value, missing value, missing value, {|name|:thisName, location:thisLocation})
end repeat
on error error_message number error_number
tell application id "DNtp" to hide progress indicator
activate
if the error_number is not -128 then display alert "JSON" message error_message as warning
return
end try
tell application "System Events"
try
my stepProgress("Creating Groups")
set thePaths to POSIX path of (disk items of folder theExportAsFolderPath whose visible is true)
my createLocations(thePaths, theExportAsFolderPath, theDatabase, theLocationAndName)
on error error_message number error_number
tell application id "DNtp" to hide progress indicator
if the error_number is not -128 then display alert "System Events" message error_message as warning
return
end try
end tell
tell application id "DNtp"
try
set theJSONExportGroup to create record with {name:"JSON Export", type:group} in theGroup
set theSmartGroup_Group to create record with {type:smart group, search predicates:"kind:group", search group:theGroup, name:"kind:group"} in theGroup
set theSmartGroup_Any to create record with {type:smart group, search predicates:"kind:any", search group:theGroup, name:"kind:any"} in theGroup
repeat with thisPath in theExportAsJSONSubfolderPaths
import thisPath to theJSONExportGroup
end repeat
my stepProgress("Creating Links ...")
set theIDs to my getPlistIDs(thePlist)
set theRecordNames_record to {}
repeat with thisID in theIDs
set theThoughtGroups to (children of theJSONExportGroup whose name = thisID)
if theThoughtGroups ≠ {} then
set theThoughtGroup to item 1 of theThoughtGroups
set theThoughtGroupChildren to (children of theThoughtGroup whose name = "Notes.md" or name = "Notes")
if theThoughtGroupChildren ≠ {} then
set theMarkdownRecord to item 1 of theThoughtGroupChildren
set {theName, theLinkIDs_outgoing, theLinkIDs_incoming, theAttachments} to {item 1, item 2, item 3, item 4} of my readPlist(thePlist, thisID)
step progress indicator ("Creating Links ... " & theName) as string
set end of theRecordNames_record to {|record|:theMarkdownRecord, |name|:theName}
if useNameAsHeading = true then set plain text of theMarkdownRecord to theHeadingLevel & space & theName & linefeed & linefeed & (plain text of theMarkdownRecord)
if theLinkIDs_outgoing ≠ {} then my writeMarkdownLinks(theLinkIDs_outgoing, theJSONExportGroup, theMarkdownRecord, "Outgoing")
if theLinkIDs_incoming ≠ {} then my writeMarkdownLinks(theLinkIDs_incoming, theJSONExportGroup, theMarkdownRecord, "Incoming")
my writeMarkdownLinks_Attachments(theAttachments, theMarkdownRecord)
end if
else
error "Can't get group ID \"" & thisID & "\""
end if
end repeat
my stepProgress("Moving Records")
repeat with this_record in theRecordNames_record
set thisRecord to (|record| of this_record)
set thisName to (|name| of this_record)
set name of thisRecord to thisName
set theGroups to (children of theSmartGroup_Group whose name = thisName)
if theGroups ≠ {} then
set thisGroup to item 1 of theGroups
set theChildren to (children of (parent 1 of thisRecord) whose type ≠ group)
if theChildren ≠ {} then
repeat with thisChild in theChildren
move record thisChild to thisGroup
end repeat
else
error "Can't move children"
end if
end if
end repeat
hide progress indicator
on error error_message number error_number
hide progress indicator
activate
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
return
end try
end tell
on existsPath(thePath)
try
exists POSIX file thePath as alias
on error
activate
display alert "Error: Handler \"existsPath\"" message "Can't get path: \"" & thePath & "\"" as warning
error number -128
end try
end existsPath
on stepProgress(theInfo)
tell application id "DNtp" to step progress indicator theInfo as string
end stepProgress
on readJSON(thePath)
try
set ndJSON_File to POSIX file thePath
set ndJSON to read ndJSON_File as «class utf8»
set theParagraphs to paragraphs of ndJSON
if (item -1 in theParagraphs) as string = "" then set theParagraphs to items 1 thru -2 in theParagraphs
set theJSON to "[" & linefeed & my tid(theParagraphs, "," & linefeed) & linefeed & "]" -- ndJSON -> JSON
set theJSON_String to current application's NSString's stringWithString:theJSON
set theJSON_Data to theJSON_String's dataUsingEncoding:(current application's NSUTF8StringEncoding) -- https://forum.latenightsw.com/t/reading-json-data-with-nsjsonserialization/958/2
set theJSON_Dictionary to current application's NSJSONSerialization's JSONObjectWithData:theJSON_Data options:0 |error|:(missing value)
on error error_message number error_number
activate
display alert "Error: Handler \"readJSON\"" message error_message as warning
error number -128
end try
end readJSON
on writePlist(theID, theName, theOutgoingLinkID, theIncomingLinkID, theAttachments)
tell application "System Events"
try
tell property list file thePlist
if not (exists property list item theID) then make new property list item at end of property list items of contents with properties {kind:record, name:theID, value:{|name|:"", |linkIDs_outgoing|:{}, |linkIDs_incoming|:{}, attachments:{}}}
if theName ≠ missing value then set (value of property list item "name" of property list item theID) to theName
set theValue_LinkIDs_outgoing to (value of property list item "linkIDs_outgoing" of property list item theID) -- testing
if theOutgoingLinkID ≠ missing value and theValue_LinkIDs_outgoing does not contain theOutgoingLinkID then set (value of property list item "linkIDs_outgoing" of property list item theID) to theValue_LinkIDs_outgoing & {theOutgoingLinkID}
set theValue_LinkIDs_incoming to (value of property list item "linkIDs_incoming" of property list item theID) -- testing
if theIncomingLinkID ≠ missing value and theValue_LinkIDs_incoming does not contain theIncomingLinkID then set (value of property list item "linkIDs_incoming" of property list item theID) to theValue_LinkIDs_incoming & {theIncomingLinkID}
set theValue_Attachments to (value of property list item "attachments" of property list item theID) -- testing
if theAttachments ≠ missing value and theValue_Attachments ≠ theAttachments then set (value of property list item "attachments" of property list item theID) to {theAttachments}
end tell
on error error_message number error_number
activate
display alert "Error: Handler \"writePlist\"" message error_message as warning
error number -128
end try
end tell
end writePlist
on createLocations(thePaths, theExportAsFolderPath, theDatabase, theLocationAndName)
tell application "System Events"
try
repeat with thisPath in thePaths
set thisPath to thisPath as string
set thisResolvedPath to my resolveAlias(thisPath)
if (class of (disk item thisResolvedPath)) = folder then
tell application id "DNtp"
try
if thisPath = thisResolvedPath then
set theLocation to my removeFromString(thisPath, theExportAsFolderPath)
create location ((theLocationAndName & "/" & theLocation) as string) in theDatabase
else
set theAliasDestination_Location to my removeFromString(thisResolvedPath, theExportAsFolderPath)
set theAliasDestination_Group to create location ((theLocationAndName & "/" & theAliasDestination_Location) as string) in theDatabase
set theAliasParent_Location to my removeFromString(my dir(thisPath), theExportAsFolderPath)
set theAliasParent_Group to create location ((theLocationAndName & "/" & theAliasParent_Location) as string) in theDatabase
replicate record theAliasDestination_Group to theAliasParent_Group
end if
on error error_message number error_number
activate
display alert "Error: Handler \"createLocations\" - DEVONthink" message error_message as warning
error number -128
end try
end tell
end if
if (class of disk item thisPath) = folder then
set thesePaths to POSIX path of (disk items of disk item thisPath whose visible is true)
my createLocations(thesePaths, theExportAsFolderPath, theDatabase, theLocationAndName)
end if
end repeat
on error error_message number error_number
activate
display alert "Error: Handler \"createLocations\"" message error_message as warning
error number -128
end try
end tell
end createLocations
on getPlistIDs(thePlist)
try
tell application "System Events"
tell property list file thePlist
set theIDs to name of property list items
end tell
end tell
on error error_message number error_number
activate
display alert "Error: Handler \"getPlistIDs\"" message error_message as warning
error number -128
end try
end getPlistIDs
on writeMarkdownLinks(theLinkIDs, theJSONExportGroup, theMarkdownRecord, theLinkSectionName)
tell application id "DNtp"
try
set theMarkdownLinks to {}
repeat with thisLinkID in theLinkIDs
set theLinkGroups to (children of theJSONExportGroup whose name = thisLinkID)
if theLinkGroups ≠ {} then
set theLinkMarkdownRecords to (children of (item 1 of theLinkGroups) whose name = "Notes.md" or name = "Notes")
if theLinkMarkdownRecords ≠ {} then
set thisLinkMarkdownRecord to item 1 of theLinkMarkdownRecords
set thisLinkMarkdownRecord_Name to item 1 of (my readPlist(thePlist, thisLinkID))
set thisLinkMarkdownRecord_ReferenceURL to reference URL of thisLinkMarkdownRecord
set end of theMarkdownLinks to ("[" & thisLinkMarkdownRecord_Name & "](" & thisLinkMarkdownRecord_ReferenceURL & ")") as string
else
error "Can't get markdown record \"" & thisLinkID & "\""
end if
else
error "Can't get group \"" & thisLinkID & "\""
end if
end repeat
if theMarkdownLinks ≠ {} then
set theMarkdownLinks_string to my tid(my sort_list(theMarkdownLinks), theLinkDelimiter)
set theMarkdownLinksSection to theLinkSectionDelimiter & linefeed & linefeed & theLinkSectionName & ": " & theMarkdownLinks_string & space & space
set theText to plain text of theMarkdownRecord
set newText to (theText & linefeed & linefeed & theMarkdownLinksSection) as string
set plain text of theMarkdownRecord to newText
end if
on error error_message number error_number
activate
display alert "Error: Handler \"writeMarkdownLinks\"" message error_message as warning
error number -128
end try
end tell
end writeMarkdownLinks
on writeMarkdownLinks_Attachments(theAttachments, theMarkdownRecord)
tell application id "DNtp"
try
set theMarkdownLinks to {}
set theAttachmentRecords to (children of (parent 1 of theMarkdownRecord) whose name ≠ "Notes.md" and name ≠ "Notes")
repeat with thisAttachmentRecord in theAttachmentRecords -- Internal
set thisAttachmentRecord_ReferenceURL to reference URL of thisAttachmentRecord
set thisAttachmentRecord_Name to name of thisAttachmentRecord
set end of theMarkdownLinks to ("[" & thisAttachmentRecord_Name & "](" & thisAttachmentRecord_ReferenceURL & ")") as string
end repeat
repeat with this_record in theAttachments -- External
set thisName to |name| of this_record
set thisLocation to |location| of this_record
if thisLocation starts with "/" then
if importExternalAttachments = false then
tell application "System Events"
try
set thisLocation to URL of disk item thisLocation
end try
end tell
set thisMarkdownLink to ("[" & thisName & "](" & thisLocation & ")") as string
set end of theMarkdownLinks to thisMarkdownLink
else
set thisAttachment to import thisLocation to (parent 1 of theMarkdownRecord)
set thisAttachment_ReferenceURL to reference URL of thisAttachment
set thisAttachment_Name to name of thisAttachment
set end of theMarkdownLinks to ("[" & thisAttachment_Name & "](" & thisAttachment_ReferenceURL & ")") as string
end if
else
set thisMarkdownLink to ("[" & thisName & "](" & thisLocation & ")") as string
set end of theMarkdownLinks to thisMarkdownLink
end if
end repeat
if theMarkdownLinks ≠ {} then
set theMarkdownLinks_string to my tid(my sort_list(theMarkdownLinks), theLinkDelimiter)
set theMarkdownLinksSection to theLinkSectionDelimiter & linefeed & linefeed & "Attachments" & ": " & theMarkdownLinks_string & space & space
set theText to plain text of theMarkdownRecord
set newText to (theText & linefeed & linefeed & theMarkdownLinksSection) as string
set plain text of theMarkdownRecord to newText
end if
on error error_message number error_number
activate
display alert "Error: Handler \"writeMarkdownLinks_Attachments\"" message error_message as warning
error number -128
end try
end tell
end writeMarkdownLinks_Attachments
on readPlist(thePlist, theID)
try
tell application "System Events"
tell property list file thePlist
if exists property list item theID then
set theName to (value of property list item "name" of property list item theID)
set theOutgoingLinkIDs to (value of property list item "linkIDs_outgoing" of property list item theID) as list
set theIncomingLinkIDs to (value of property list item "linkIDs_incoming" of property list item theID) as list
set theAttachments to (value of property list item "attachments" of property list item theID) as list
return {theName, theOutgoingLinkIDs, theIncomingLinkIDs, theAttachments}
else
error "Plist: Can't get ID \"" & thisID & "\""
end if
end tell
end tell
on error error_message number error_number
activate
display alert "Error: Handler \"readPlist\"" message error_message as warning
error number -128
end try
end readPlist
on tid(theInput, theDelimiter)
set d to AppleScript's text item delimiters
set AppleScript's text item delimiters to theDelimiter
if class of theInput = text then
set theOutput to text items of theInput
else if class of theInput = list then
set theOutput to theInput as text
end if
set AppleScript's text item delimiters to d
return theOutput
end tid
on resolveAlias(thePath)
try
set thePath_URL to (current application's class "NSURL"'s fileURLWithPath:thePath)
set thePath_Original_URL to (current application's class "NSURL"'s URLByResolvingAliasFileAtURL:thePath_URL options:0 |error|:(missing value))
set thePath_Original to (POSIX path of (thePath_Original_URL as string))
if thePath_Original ends with "/" then set thePath_Original to (characters 1 thru -2 in thePath_Original) as string
return thePath_Original
on error error_message number error_number
activate
display alert "Error: Handler \"resolveAlias\"" message error_message as warning
error number -128
end try
end resolveAlias
on dir(thePath)
try
set revPath to (reverse of characters of thePath) as string
return (reverse of (characters ((offset of "/" in revPath) + 1) thru -1 in revPath)) as string
on error error_message number error_number
activate
display alert "Error: Handler \"dir\"" message error_message as warning
error number -128
end try
end dir
on removeFromString(theText, CharOrString)
local ASTID, theText, CharOrString, lst
set ASTID to AppleScript's text item delimiters
try
considering case
if theText does not contain CharOrString then ¬
return theText
set AppleScript's text item delimiters to CharOrString
set lst to theText's text items
end considering
set AppleScript's text item delimiters to ASTID
return lst as text
on error eMsg number eNum
set AppleScript's text item delimiters to ASTID
error "Can't RemoveFromString: " & eMsg number eNum
end try
end removeFromString
on sort_list(theList)
considering numeric strings
set theIndexList to {}
set theSortedList to {}
repeat (length of theList) times
set theLowItem to ""
repeat with a from 1 to (length of theList)
if a is not in theIndexList then
set theCurrentItem to item a of theList as text
if theLowItem is "" then
set theLowItem to theCurrentItem
set theLowItemIndex to a
else if theCurrentItem comes before theLowItem then
set theLowItem to theCurrentItem
set theLowItemIndex to a
end if
end if
end repeat
set end of theSortedList to theLowItem
set end of theIndexList to theLowItemIndex
end repeat
end considering
return theSortedList
end sort_list
@pete31 thanks, I’ll test this later.
One thing I could change, or make optional by commented code, would be to import the data to a new database. Obviously, we can create a database and do that if we want that option. Some “brains” can be very large and moving the import into a new database for cleanup might be a good idea. With DEVONthink 3, moving the import over to existing database(s), post-cleanup, is no big deal.
Unfortunately JSON export in TheBrain is all of nothing – you cannot export a subset of a database.
Thought of it while writing but finally skipped it as it didn’t make sense in my case with this little test brain, for large brains this is of course a good idea.
It’s possible to export the desired subset as “Folders” export and then match the JSON against it. That’s how records are moved in the script too, so exporting subsets should be doable.
Sure, for someone who wants to muck around in different exports, munge them together, and figure it out. Ordinary folk don’t need that annoyance. But, my point is that TheBrain is not designed to merely select several “thoughts” (notes) and export just that selection to JSON. IMO, it’s bad design, but the product manager of TheBrain does not agree with me.
1 Like
Ill give it a spin over the week-end.
1 Like
Tried exporting my Brain 3389 thoughts, 3509 links (TheBrain version 11.0.131)
JSON : no probs; creates what seems to be a “correct” export.
folders : export failed! invalid file or directory attribute value.
So I couldn’t run the test.
pete31: I appreciate your effort and the time you have put in, and wouldn’t want to appear as complaining, but why do you need a “folder” export in addition to the JSON export? Seems to me -but I realize I haven’t done the work- that we could manage without the folder export, as everything seem to be available in the JSON export : thoughts, links and attachments.
I thought the script failed 
Export as folders with TheBrain 11.0.131 works fine over … That’s exactly why I want to know whether it would work with real data …
BTW, korm had problems with Export as folders too:
Here’s a new script.
It doesn’t need folders export anymore, just JSON.
@ClaudeB, @korm, @shiiko you might want to try this.
-- Import TheBrain's JSON export
-- Create a new folder, export as JSON
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
property useNameAsHeading : true
property theHeadingLevel : "###"
property theLinkSectionDelimiter : " "
property theLinkDelimiter : " "
property importExternalAttachments : false
global thePlist
tell application "System Events"
try
activate
set theExportAsJSON_Path to POSIX path of (choose folder with prompt "Choose JSON export folder:" default location (path to desktop folder) without invisibles)
set theSubfolderPaths to POSIX path of folders of folder theExportAsJSON_Path
if theSubfolderPaths = {} then error "Error: Missing \"JSON\" export"
set theThoughtsJSON_Path to (theExportAsJSON_Path & "/" & "thoughts.json") as string
my existsPath(theThoughtsJSON_Path)
set theLinksJSON_Path to (theExportAsJSON_Path & "/" & "links.json") as string
my existsPath(theLinksJSON_Path)
set theAttachmentsJSON_Path to (theExportAsJSON_Path & "/" & "attachments.json") as string
my existsPath(theAttachmentsJSON_Path)
set thePlist to (theExportAsJSON_Path & "/" & "_Brain.plist") as string
try
exists POSIX file thePlist as alias
on error
make new property list file with properties {name:thePlist}
end try
on error error_message number error_number
activate
if the error_number is not -128 then display alert "System Events" message error_message as warning
return
end try
end tell
tell application id "DNtp"
try
activate
set theGroup to display group selector
set theDatabase to database of theGroup
show progress indicator "Importing Brain ..." steps 4
on error error_message number error_number
activate
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
return
end try
end tell
try
my stepProgress("Reading JSON ...")
set theJSON_LinksDictionary to my readJSON(theLinksJSON_Path)
repeat with thisItem in theJSON_LinksDictionary
if (|Meaning| of thisItem) as integer = 1 then
set thisThoughtIdA to (|ThoughtIdA| of thisItem) as string
set thisThoughtIdB to (|ThoughtIdB| of thisItem) as string
my writePlist(thisThoughtIdA, missing value, thisThoughtIdB, missing value, missing value)
my writePlist(thisThoughtIdB, missing value, missing value, thisThoughtIdA, missing value)
end if
end repeat
set theJSON_ThoughtsDictionary to my readJSON(theThoughtsJSON_Path)
repeat with thisItem in theJSON_ThoughtsDictionary
if (|Kind| of thisItem) as integer = 1 then
my writePlist((|Id| of thisItem) as string, (|Name| of thisItem) as string, missing value, missing value, missing value)
end if
end repeat
set theJSON_AttachmentsDictionary to my readJSON(theAttachmentsJSON_Path)
repeat with thisItem in theJSON_AttachmentsDictionary
set thisName to (|Name| of thisItem) as string
set thisLocation to (|Location| of thisItem) as string
if thisName ≠ thisLocation then
my writePlist((|SourceId| of thisItem) as string, missing value, missing value, missing value, {|name|:thisName, location:thisLocation})
end if
end repeat
on error error_message number error_number
tell application id "DNtp" to hide progress indicator
activate
if the error_number is not -128 then display alert "JSON" message error_message as warning
return
end try
tell application id "DNtp"
try
set theJSONExportGroup to create record with {name:"Brain", type:group} in theGroup
set theSmartGroup_Group to create record with {type:smart group, search predicates:"kind:group", search group:theGroup, name:"kind:group"} in theGroup
set theSmartGroup_Any to create record with {type:smart group, search predicates:"kind:any", search group:theGroup, name:"kind:any"} in theGroup
my stepProgress("Importing folders ...")
repeat with thisPath in theSubfolderPaths
import thisPath to theJSONExportGroup
end repeat
my stepProgress("Writing links ...")
set theIDs to my getPlistIDs(thePlist)
set theRootID to missing value
set the_record to {}
repeat with thisID in theIDs
set thisID to thisID as string
set {theName, theLinkIDs_outgoing, theLinkIDs_incoming, theAttachments} to {item 1, item 2, item 3, item 4} of my readPlist(thePlist, thisID)
if theLinkIDs_incoming = {} then set theRootID to thisID
set theThoughtGroups to (children of theJSONExportGroup whose name = thisID)
if theThoughtGroups ≠ {} then
set theThoughtGroup to item 1 of theThoughtGroups
else
set theThoughtGroup to create record with {name:thisID, type:group} in theJSONExportGroup
end if
repeat with thisAttachment in theAttachments
set thisLocation to |location| of thisAttachment
if thisLocation starts with "/" then
if importExternalAttachments = false then
tell application "System Events"
try
set thisLocation to (URL of disk item thisLocation) as string
end try
end tell
create record with {type:bookmark, URL:thisLocation, name:(|name| of thisAttachment)} in theThoughtGroup
else
import thisLocation to theThoughtGroup
end if
else
create record with {type:bookmark, URL:thisLocation, name:(|name| of thisAttachment)} in theThoughtGroup
end if
end repeat
set theThoughtGroupChildRecords to (children of theThoughtGroup whose name = "Notes.md" or name = "Notes")
if theThoughtGroupChildRecords ≠ {} then
set theMarkdownRecord to item 1 of theThoughtGroupChildRecords
if useNameAsHeading = true then set plain text of theMarkdownRecord to theHeadingLevel & space & theName & linefeed & linefeed & (plain text of theMarkdownRecord)
step progress indicator ("Writing Links ... " & theName) as string
if theLinkIDs_outgoing ≠ {} then my writeMarkdownLinks(theLinkIDs_outgoing, theJSONExportGroup, theMarkdownRecord, "Outgoing")
if theLinkIDs_incoming ≠ {} then my writeMarkdownLinks(theLinkIDs_incoming, theJSONExportGroup, theMarkdownRecord, "Incoming")
my writeMarkdownLinks_Attachments(theMarkdownRecord, theThoughtGroup)
else
set theMarkdownRecord to missing value
end if
set end of the_record to {|id|:thisID, |outgoingLinks|:theLinkIDs_outgoing, |incomingLinks|:theLinkIDs_incoming, |group|:theThoughtGroup, |record|:theMarkdownRecord, |name|:theName}
end repeat
if theRootID ≠ missing value then
my stepProgress(("Moving Records ... " & theName) as string)
set theRootGroups to (children of theJSONExportGroup whose name = theRootID)
if theRootGroups ≠ {} then set theRootGroup to item 1 of theRootGroups
repeat with this_record in the_record
if theRootID ≠ (|id| of this_record) then
repeat with thisLinkId_outgoing in (|outgoingLinks| of this_record)
set theGroups to (children of theJSONExportGroup whose name = (thisLinkId_outgoing as string))
if theGroups ≠ {} then
replicate record (item 1 of theGroups) to (|group| of this_record)
end if
end repeat
end if
end repeat
repeat with this_record in the_record
set theThoughtGroup to |group| of this_record
set name of theThoughtGroup to |name| of this_record
set theRecord to (|record| of this_record)
if theRecord ≠ missing value then set name of theRecord to |name| of this_record
if theRootID ≠ (|id| of this_record) then
if (|incomingLinks| of this_record) does not contain theRootID then
if (number of replicants of theThoughtGroup) > 0 then
delete record theThoughtGroup in theJSONExportGroup
end if
else
move record theThoughtGroup from theJSONExportGroup to theRootGroup
end if
end if
end repeat
end if
hide progress indicator
on error error_message number error_number
hide progress indicator
activate
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
return
end try
end tell
on existsPath(thePath)
try
exists POSIX file thePath as alias
on error
activate
display alert "Error: Handler \"existsPath\"" message "Can't get path: \"" & thePath & "\"" as warning
error number -128
end try
end existsPath
on stepProgress(theInfo)
tell application id "DNtp" to step progress indicator theInfo as string
end stepProgress
on readJSON(thePath)
try
set ndJSON_File to POSIX file thePath
set ndJSON to read ndJSON_File as «class utf8»
set theParagraphs to paragraphs of ndJSON
if (item -1 in theParagraphs) as string = "" then set theParagraphs to items 1 thru -2 in theParagraphs
set theJSON to "[" & linefeed & my tid(theParagraphs, "," & linefeed) & linefeed & "]" -- ndJSON -> JSON
set theJSON_String to current application's NSString's stringWithString:theJSON
set theJSON_Data to theJSON_String's dataUsingEncoding:(current application's NSUTF8StringEncoding) -- https://forum.latenightsw.com/t/reading-json-data-with-nsjsonserialization/958/2
set theJSON_Dictionary to current application's NSJSONSerialization's JSONObjectWithData:theJSON_Data options:0 |error|:(missing value)
on error error_message number error_number
activate
display alert "Error: Handler \"readJSON\"" message error_message as warning
error number -128
end try
end readJSON
on writePlist(theID, theName, theOutgoingLinkID, theIncomingLinkID, theAttachments)
tell application "System Events"
try
tell property list file thePlist
if not (exists property list item theID) then make new property list item at end of property list items of contents with properties {kind:record, name:theID, value:{|name|:"", |linkIDs_outgoing|:{}, |linkIDs_incoming|:{}, attachments:{}, |kind|:0}}
if theName ≠ missing value then set (value of property list item "name" of property list item theID) to theName
set theValue_LinkIDs_outgoing to (value of property list item "linkIDs_outgoing" of property list item theID)
if theOutgoingLinkID ≠ missing value then set (value of property list item "linkIDs_outgoing" of property list item theID) to theValue_LinkIDs_outgoing & {theOutgoingLinkID}
set theValue_LinkIDs_incoming to (value of property list item "linkIDs_incoming" of property list item theID)
if theIncomingLinkID ≠ missing value then set (value of property list item "linkIDs_incoming" of property list item theID) to theValue_LinkIDs_incoming & {theIncomingLinkID}
if theAttachments ≠ missing value then set (value of property list item "attachments" of property list item theID) to {theAttachments}
end tell
on error error_message number error_number
activate
display alert "Error: Handler \"writePlist\"" message error_message as warning
error number -128
end try
end tell
end writePlist
on getPlistIDs(thePlist)
try
tell application "System Events"
tell property list file thePlist
set theIDs to name of property list items
end tell
end tell
on error error_message number error_number
activate
display alert "Error: Handler \"getPlistIDs\"" message error_message as warning
error number -128
end try
end getPlistIDs
on readPlist(thePlist, theID)
try
tell application "System Events"
tell property list file thePlist
if exists property list item theID then
set theName to (value of property list item "name" of property list item theID)
set theOutgoingLinkIDs to (value of property list item "linkIDs_outgoing" of property list item theID) as list
set theIncomingLinkIDs to (value of property list item "linkIDs_incoming" of property list item theID) as list
set theAttachments to (value of property list item "attachments" of property list item theID) as list
return {theName, theOutgoingLinkIDs, theIncomingLinkIDs, theAttachments}
else
error "Plist: Can't get ID \"" & thisID & "\""
end if
end tell
end tell
on error error_message number error_number
activate
display alert "Error: Handler \"readPlist\"" message error_message as warning
error number -128
end try
end readPlist
on writeMarkdownLinks(theLinkIDs, theJSONExportGroup, theMarkdownRecord, theLinkSectionName)
tell application id "DNtp"
try
set theMarkdownLinks to {}
repeat with thisLinkID in theLinkIDs
set theLinkGroups to (children of theJSONExportGroup whose name = thisLinkID)
if theLinkGroups ≠ {} then
set theLinkMarkdownRecords to (children of (item 1 of theLinkGroups) whose name = "Notes.md" or name = "Notes")
if theLinkMarkdownRecords ≠ {} then
set thisLinkMarkdownRecord to item 1 of theLinkMarkdownRecords
set thisLinkMarkdownRecord_Name to (item 1 of (my readPlist(thePlist, thisLinkID))) as string
set thisLinkMarkdownRecord_ReferenceURL to (reference URL of thisLinkMarkdownRecord) as string
set end of theMarkdownLinks to ("[" & thisLinkMarkdownRecord_Name & "](" & thisLinkMarkdownRecord_ReferenceURL & ")") as string
end if
end if
end repeat
if theMarkdownLinks ≠ {} then
set theMarkdownLinks_string to my tid(my sort_list(theMarkdownLinks), theLinkDelimiter)
set theMarkdownLinksSection to (theLinkSectionDelimiter & linefeed & linefeed & theLinkSectionName & ": " & theMarkdownLinks_string & space & space) as string
set theText to plain text of theMarkdownRecord
set plain text of theMarkdownRecord to (theText & linefeed & linefeed & theMarkdownLinksSection) as string
end if
on error error_message number error_number
activate
display alert "Error: Handler \"writeMarkdownLinks\"" message error_message as warning
error number -128
end try
end tell
end writeMarkdownLinks
on writeMarkdownLinks_Attachments(theMarkdownRecord, theThoughtGroup)
tell application id "DNtp"
try
set theMarkdownLinks to {}
set theAttachmentRecords to (children of theThoughtGroup whose name ≠ "Notes.md" and name ≠ "Notes")
repeat with thisAttachmentRecord in theAttachmentRecords
set thisAttachmentRecord_ReferenceURL to (reference URL of thisAttachmentRecord) as string
set thisAttachmentRecord_Name to (name of thisAttachmentRecord) as string
set end of theMarkdownLinks to ("[" & thisAttachmentRecord_Name & "](" & thisAttachmentRecord_ReferenceURL & ")") as string
end repeat
if theMarkdownLinks ≠ {} then
set theMarkdownLinks_string to my tid(my sort_list(theMarkdownLinks), theLinkDelimiter)
set theMarkdownLinksSection to (theLinkSectionDelimiter & linefeed & linefeed & "Attachments" & ": " & theMarkdownLinks_string & space & space) as string
set theText to plain text of theMarkdownRecord
set plain text of theMarkdownRecord to (theText & linefeed & linefeed & theMarkdownLinksSection) as string
end if
on error error_message number error_number
activate
display alert "Error: Handler \"writeMarkdownLinks_Attachments\"" message error_message as warning
error number -128
end try
end tell
end writeMarkdownLinks_Attachments
on tid(theInput, theDelimiter)
set d to AppleScript's text item delimiters
set AppleScript's text item delimiters to theDelimiter
if class of theInput = text then
set theOutput to text items of theInput
else if class of theInput = list then
set theOutput to theInput as text
end if
set AppleScript's text item delimiters to d
return theOutput
end tid
on sort_list(theList)
considering numeric strings
set theIndexList to {}
set theSortedList to {}
repeat (length of theList) times
set theLowItem to ""
repeat with a from 1 to (length of theList)
if a is not in theIndexList then
set theCurrentItem to item a of theList as text
if theLowItem is "" then
set theLowItem to theCurrentItem
set theLowItemIndex to a
else if theCurrentItem comes before theLowItem then
set theLowItem to theCurrentItem
set theLowItemIndex to a
end if
end if
end repeat
set end of theSortedList to theLowItem
set end of theIndexList to theLowItemIndex
end repeat
end considering
return theSortedList
end sort_list
Many thanks Pete
The script failed after running for approx 35 minutes !!!
Error message was (locale language is french so it’s a bit mixed up) :
Impossible de convertir |Name| of item 3 of «class ocid» id «data optr0000000000D4A4FDF77F0000» en type string.
Wow. What is the result you have in DEVONthink now, if any? Were groups created? If so, do they have cryptic names or the names of your Thoughts?
I’m afraid nothing was created because this
means it was not possible to coerce a part of the JSON to an AppleScript string. The part where this is done is quite at the beginning of the script, which is strange as I think it’s impossible that the script took 35 minutes til the part that I assume to be the only one that throws such an error.
I have ideas on how to deal with this, but I first need to know what was created in DEVONthink. Also wrote to TheBrain support.
May I suggest that the structure of information in The Brain, while extremely useful, does not really mesh well with the data structures in DevonThink. It’s sort of like translating French Literature into scientific notation --> does not compute.
I would suggest two alternatives:
(1) Print the contents of your Brain projects to PDF and then import those PDFs into Devonthink.
(2) Keep your data in The Brain just as it is. Then just use DT3 to index the folder in which you keep your Brain projects. To view a Brain project, right click on it within DT3 and choose “Open With… The Brain.”
@rkaplan I tried what you suggested
- That’s not possible. Printing is not available in TheBrain’s macOS version.
- That’s not possible. Opening a TheBrain database from within Finder or DEVONthink is not possible.
Even if opening a database would be possible indexing would result in
- cryptic group names (name = Thought ID)
- text would be searchable but all markdown files are named “Notes.md”
- no (internal) attachments
- no links
- no group hierachy
That’s no way to go, at least not for me.
The script does work over here and results in
- properly named records and groups,
- completely searchable content, including attachments
- included attachments
- incoming links, outgoing links, links to attachments
- group hierachy
We “just” have to find out why it’s failing with Claude’s data. 
Yeah those do not work well - sorry
How about the “Folder Import” method that Brain supports?
Which takes us back again to the discussion of folder imports at the beginning of the the thread.
1 Like
Hi and thanks to all for your inputs and suggestions.
I’ve played around with the script from Pete, looked at the JSON data as exported from TBn and basically gotten clear about what I want to do. Here’s where I’m at, Caveat : I used to code, but that was a looooooong time ago.
TheBrain’s unit is a thought.
A thought can have a type (only one type -which is good- but not necessarily one, thoughts can be untyped).
A thought has a note attached to it which contains text (RTF? Markdown? plain text? don’t know.
A thought can have tags, unlimited number of them as far as I know
A thought can have attachments (files and URLs), unlimited as far as I know
A thought can have links to other thoughts, unlimited number of them. Links have a direction (from-to) and can have tags, attachments? I don’t know I don’t use this feature.
In the thoughts.json file exported by therBrain, the value of the key “Kind” identifies thoughts (1), types(2) and tags(4)
Up to now, I used Thebrain as an active repository of “market” information : people, organizations, products, brands, features, and information relative to these items such as press releases.
ex : People are thoughts with type “people”.
I have 27 diefferent types, it’s been stable for a long while.
I have 80+ tags, and this increases over time.
I am moving to Devonthink, for reasons too long to explain here. I want to migrate my data from ThB to DT.
In DT I’d like to see
- TheBrain Type of thoughts should be DT groups ie. groups named People, Organizations, etc. with the “thoughts” directly under according to their Type . Untyped thoughts should be placed in a dedicated “untyped” group.
- Notes of a thought should be directly in the DT item’s description field
- Tags of theBrain map naturally to the same in DT.
- Attachments should be in a separate group, or directly under the Brain database.
- Links with thoughts could be in the bottom of the corresponding DT item, something like a footer (in : … and out…)
I’ve tried to set this up using a modified version of Pete’s script, but am far from succeeding as I am lacking knowledge about AppleScript (i.e. don’t know how to code the case where there is no key for TypeId -untyped thoughts-).
Nevertheless, I’ll stick with it, find out and keep you posted.
@ClaudeB, I believe you’re looking for a custom script, used one-time, to setup a database in a very particular way that fits the design of the brain database that you want to migrate from. I applaud @pete31’s amazing volunteer effort to provide a generic brain-to-DEVONthink import script. You will not find the specific script you’re looking for, unless someone devotes a lot of time volunteer the answer. A solution that would be used one time then discarded.
Even then, you cannot have exactly the structure in DEVONthink you want.
- There is no built-in design in DEVONthink corresponding to TheBrain’s 'Type" and “Tags” – you can accomplish this, however, by expanding your understand of what tags in DEVONthink can accomplish, and adding custom metadata to your database.
- “Notes of a thought should be directly in the DT item’s description field” – I assume you mean you want the notes field of a thought in TheBrain to be imported into the “comment” field of a document in DEVONthink? I would suggest that you’ll have a mess on your hands trying to work with your data if you do that. The comment field is meant for a small amount of text.
- “Attachments should be in a separate group, or directly under the Brain database.” What does that mean, exactly? Don’t you want your attachments to be related to your notes, as they are in TheBrain?
- “Links with thoughts could be in the bottom of the corresponding DT item, something like a footer (in : … and out…)” Again, what does that mean? What is your understanding of a “DT item”? DEVONthink contains files – just files – of almost any kind.
Instead of trying to make DEVONthink behave like TheBrain, I suggest stepping back and thinking “ok, I have this information structure in TheBrain – so what kind of information structure can I have within the design constraints of DEVONthink that enables me to continue working with my data”.
It’s going to be a manual effort for the most part. You might be better off exporting the folders from TheBrain, importing them in their raw form into DEVONthink, and working to rebuild the structure you want.
TheBrain advertises that it provides portability of our data. It does not! As you discovered, once we use Types and Tags and Links and events and (now in v12) Mentions and Backlinks – the lock-in to TheBrain’s structure is tight, with no way out.
That’s not necessarily a bad thing – just needs awareness.
2 Likes
There is another possible answer - this one I tried and does work well
You can use Hook - it works well with Brain 12 to create a link to any given portion of a Brain project
It creates a link similar to this format:
brain://api.thebrain.com/k6Pjl7MXg0KcValR3OL4aQ/RXpTxBnWb1CmBDLl3v8MJg/InspirationalIdeas
You can then enter that link as a Bookmark in DT3
Then a simply click of that item in DT3 instantly opens up TheBrain to the desired location