Synchronising a Devonthink database with 2 computers

I have used both the Apple Migration Assistant and Chronosync to get Devonthink across to my new Intel iMac (10.4.8) from my Powerbook G4 (10.4.8). Migration assistant gets the Devonthink app across to the iMac but the database is empty when I subsequently launch the Devonthink app on the iMac. The database backups appeared to be transported correctly across to the iMac. Then I tried using Chronosync to synchronise the home folders of the two computers; the database remained empty when Devonthink 1.9.7 was relaunched on the iMac after the sychronisation was completed. It is possible to manually import the backup database into Devonthink on the iMac but I need the database transferred from the G4 to load automatically into DT when this is launched on the iMac; I like to think that I should then be able to perform a two-way synchronisation of my Devonthink database on both the laptop and the desktop on a regular basis. The failure of Devonthink to automatically load the database after I used both Migration Assistant and Chronosync is a real concern. I don’t have a problem with whole volume backups - use SuperDuper! for that.

Suggestions as to how this sync problem might be resolved would be most welcome.

[1] ChronoSync cannot read the monolithic database body in DEVONthink. I would not trust it to do two-way synchronization of the databases on the two computers.

If changes have been made on both databases, I would use Tools > History to select the recent changes on each machine, export them to a target folder created in the Finder to hold each export, then import the contents into each of the respective databases.

[2] Your database is located at ~/Library/Application Support/ in the DEVONthink folder.

To transfer the database from one computer to the other, simply select (don’t open) the DEVONthink folder and copy it to the corresponding position on the other computer. If an empty database already exists in the target location (created, for example, by having launched DEVONthink) it should first be removed, or replaced by the copy transferred from the original computer.

Thanks for the prompt reply Bill. Since Chronosync gets good reviews and appears to be popular, just like Devonthink, it would be great if Devontech and Econtech could get together to sort out this difficulty, since I consider your proposed solution to be too time-consuming for everyday use. I suspect that other users of Devonthink who wish to use synchronising software on a small home LAN might like a straight-forward “Mac-like” solution to this current incompatibility, which in my view is a significant one. I’d be interested to see whether any other members of the forum feel the same way.

Btw: do you know of any synchronising software that does work with Devonthink databases.

It’s not an issue of being “Mac-like” or not. :slight_smile:

Most 2-way synchronization applications compare the versions of files, such as text files or whatever and pick the most recently modified one. (And sometimes that simple technique doesn’t work. There are logical possibilities in 2-way synchronization that can result in loss of data. Sometimes I’d rather see the synch software present two versions of a file rather than pick just one.) In other words, I’ve never entirely trusted 2-way synchronization, even for simple cases of ordinary files in the Finder.

The current DEVONthink database holds text, HTML and some other files in a “monolithic” database structure that synchronizing applications can’t interpret. Picking the most recently modified database won’t cut it.

There’s a third-party script http://www.woodenbrain.com/index-sw.html that is designed to synchronize two DT Pro databases over a local network. I haven’t had a chance to evaluate it. But it won’t work for DT PE.

I’ve used the History technique many times to synchronize 2 databases, both DT PE and DT Pro. As I usually work on one computer or the other, it’s pretty quick and easy to identify and export the new material from the one I’ve been working on, and send it to the other computer’s database. Even in cases where I’ve done content changes on both databases I’ve been able to synchronize the two in minutes.

Hi Bill,

sorry if I jump in the conversation…

At first I was trying the same thing using Chronosync, of course knowing that the software would prompt me to select the database file as a whole.

I tried avoiding that by simply exporting the folders on the iDisk and the import them on the other computer but with not a lot of success:
the downside of this method seems that I cannot “rewrite” the files and the folders on the same target. Files that I delete or rename will not be changed, and even worse a good portion of the links don’t work anymore when imported.

Any suggestion on how to solve the issue without using the log? I have to agree that sound pretty long when dealing with 20+ files per day.
Would the situation change using DT Pro? I mean by creating different databases and then just linking them?

Thank You for the support.
Marco

maquilab, you have run into still another complication in attempting to maintain the same database on more than one computer.

If the database contains files that have been captured using the Index mode, moving the database will break the paths to the externally linked files and information will be lost. The externally linked files will also have to be copied to the other computer, and if the paths are broken they would have to be edited to allow the database to “see” them.

Most of my databases were created by Importing files, so that they are actually copied into the database package. That makes it a much simpler matter to move the databases among different computers while maintaining complete access to the information in the database.

Note: Remember that in the current version of DT Pro, MS Word files remain externally linked even if captured in the Import mode. If the database is moved to a different computer, the database still contains the rich text content of the Word file, but cannot access the original file unless it had also been copied to the second computer, with a working Path to the file from the database. In version 2.0 this pesky exception will no longer be true; Word files can then be Imported and copied into the database.

The above comments about the complications caused by Index captures for movement of a database from one computer to another, or for updating changes to a database used on more than one computer apply both for DT PE and DT Pro. In other words, if you have added externally linked files to one of the databases, you will have to transfer to the other computer both the information content for those additions (I use Tools > History to do that) and those externally linked files – and the transfer should be such that the Paths are not broken (else the Paths must be edited in order to access the transferred external files).

And that, in a nutshell, is why most of my databases are self-contained. I use Import to capture files from the Finder. Once I’ve captured those files, I no longer need the original files and will delete them, or perhaps archive them. I have absolutely no interest in any correspondence between the content of my database and the organization and content of files in the Finder. Of course, I make external backups of my database so that I don’t have to worry about losing data. DT Pro’s Scripts > Export > Backup Archive is great for that purpose.

The downside of making Import captures of files into the database is somewhat higher memory usage (that downside will go away in version 2.0). The advantage is portability of the database. Just copy it to another computer and open it. Everything immediately works (with the exception of Word files, which still must be copied over to the other computer in a way that doesn’t break their paths).

Hi Bill,
thank you for your help, yes, I always import everything. So all the documents are contained in the database. I don’t index files (because I would have to adjust the path/directory in each computer) and given the MS Word limitation of the current version I use the internal editor. So I guess all files are in rtfd.

There are a couple of clarification though:
I have backed-up and optimized the database, then when moving the database files (9 with .database extension, 1 with .lock extension and the files folder) to the other computer for the sync it resulted in a loss of data. ( a lot of pictures and other text files and pdf have now 0 bytes and are in blue)
In detail I have 3 different backups folder since I told devonthink to mantain 3 backups archive. Prior to sync I checked the creation date and left only the latest one (the one with the lowest number after the extension). I imagined devonthink would act as follows: the file folders contains the CURRENT files, while in the backup folders there are the devonthink database files and the eventually imported files that have been removed or renamed. Is it correct?

What am I doing wrong? Why did I experience this loss of data?

This might have already been ansked somewhere else but…
Also, I usually link presentation files .key from a word in a rich text document.
(I know that it’s an unknown format for DT)
when I click on it I would expect the realative application to start and open such fil. Instead a DT windows opens, there I onley see a white background and I cannot edit it or add text. If I want to open the file I have to click on show info and then click on the @ symbol, and finally that file is opened in the relative application. What’s the utility of such a process?
Also why we cannot make a link from an image to whatever? These functions would be great when dealing with contacts!

Again thank you

Not quite. The DEVONthink PE database, which is found at ~/Library/Application Support/ is the folder named DEVONthink.

Inside that folder are 10 numbered files with the suffix “.database”, a Files folder that contains imported (copied) file types such as PDF, postscript, image and QuickTime media files, and the internal Backup folders (the default is 3).

The “body” of the database consists of the 10 numbered files and contains metadata and certain content file types, including text files (plain text, RTF, RTFD, tab-delimited text, comma-delimited text, etc., XML, OPML, HTML and WebArchive files. So this monolithic body of the database contains the metadata for all document contents (including the documents stored in the Files folder), the organizational structure information, and those text-type contents. Example: My main database contains about 21,000 documents, of which more than 15,000 are text-type files stored in the “body” of the database, so that database body is quite large in my case.

Here’s the problem for incremental backup software: Suppose today I make only one change in my database, creating a new RTF file that contains only 1 KB of information. But the backup software can’t look inside the body of the database and see that was the only change. Instead, it can only detect that the body has been changed, and will have to copy the entire body over to the backup medium. Instead of an incremental change of only 1 KB, perhaps hundreds of MB will have to be copied. Obviously, 2-way synchronization would be even more problematic, unless the synchronization software can actually read and “understand” the database. There’s a third-party application/script called DevonSynch that I’ve not evaluated, but may be able to synchronize two versions of a DT Pro database, as it is designed to “understand” the database structure.

OK, what are those internal Backup folders, “Backup”, “Backup0” and “Backup1”? Each folder contains an earlier state of the database body. They are a safety net. If the database were to become corrupt, one could use Tools > Restore Backup to switch to the state of the database as of the most recent backup, contained in “Backup”. If that database were also corrupt because the problem has existed for some time, perhaps “Backup0” contains a good, working database, and so on.

If you are going to move the database to another computer, I would suggest you simply move the entire folder to the corresponding position on the other computer, i.e. ~/Library/Application Support/.

Cautions: Make certain that DEVONthink has been quit, so that the database is not open at the time the copy is made. Copying an open database could result in a defective copy. Also, make certain that the target computer has sufficient free drive space to manage the temporary files that may require drive space for the application system and, of course, temporary files created by the operating system – allow a minimum of about 10-15% of the HD capacity as free space.

If the copy of the database were made while it was open, problems could result.

If some of the files were not in fact contained inside the database folder, but were externally linked, information would be lost.

If the database had problems before it was copied, those problems would also show up in the copy. Recommendation: Before making a copy, make certain that the database on the original computer is in good shape. Run Tools > Verify, followed (if no error report) by Backup & Optimize. Now quit DEVONthink, copy the DEVONthink database folder and move it to the same relative location on the second computer. Note: The name of the database folder must be “DEVONthink” (without quotes). Launch DEVONthink on that computer and examine the database. If you still have problems, send a message to Support.

For DT Pro users, it’s easier to simply copy a database from one computer to another. I often make the transfer file smaller by running DT Pro Scripts > Export > Backup Archive. The Backup Archive script first runs the quality assurance routines, Verify and Backup & Optimize, then saves to a chosen destination the smallest possible compressed and dated archive of the database.

As I understand this, you have set Preferences > Import to import “unknown” file types to your database files folder. So your presentation file (perhaps Powerpoint or Keynote) is actually contained in your database. This results in the creation of a “blank” document in your database, since DT can’t read text from the unknown file; the document contains only metadata about the file such as name, file type, creation and modification date, etc.

Now, in a rich text file, you create a hyperlink to that presentation file, using either a “Link To” contextual option for selected text, or a Wiki link for the presentation file’s name. That’s convenient if you wish to bring up the “blank” document corresponding to the presentation file, then click on Launch Path to open it under its parent application.

That “2-step” access from a rich text link could be shortened to one step if you Command-Option-Drag & Drop a file from the Finder to the insertion point in your DT rich text document. Clicking on that link will directly open the file under it’s parent application.

That’s currently not possible because links can only be established in a rich text document. So you can link to, but not from an image, PDF or QuickTime file. Likewise, you can link to, but not from a plain text document such as a Sheet or Record.

But take a look at today’s Tuesday tip in Eric’s blog at http://www.devon-technologies.com/scripts/wordpress/. It’s possible to use custom icons for documents and groups.

Sorry for the long-winded explanations in some of the responses, but hope they help.

Hi Bill,
I’m pretty new to DT and all this info are pretty useful. Thank You a lot!
Can you confirm the following:
About the 2way sync, that’s exactly what I’m doing with Chronosync, I now know I have to sync even before opening DT or CS might interpret a file change. I’m a PE user, so for the time being the shareware program you suggested (Devonsync) might be a solution after an eventual upgrade.
As I’m doing it the syncing becomes a 1way sync from the newer to the older file. This could definitely involve a big file exchange but if that is the only problem I think it’s still duable, also, if the database change involves only one or a few files CS will notice that and will skip the others. GOOD TO KNOW: the ENTIRE DEVONthink folder.:8) (Thank You for the clarifications on the backup folders, the 10 .database files etc.)
At this point I should use the export command only for “publishing the database” or eventually using it as an archive copy for the FILES ONLY since the rest of the database is NOT there. It’s in my application support folder!
I tried running “verify” and I did have 40 errors on the db, unfortunately it didn’t recover any data…probably my mistake in copying over the files.

In so doing, I index the file to the database, it doesn’t get exported. Instead, when I simply drag and drop it in a rich text document, it gets copied in the file folder and I can still launch it directly (this gives me the advante of ignoring the directory/path etc when sync the two computers) can you please confirm it?

It’s useful for personalizing icons and stuff… I’m studying Management and Economics and often I have to use curves and other graphic elements, so it could be great to have a picture of small size within the text, click on it and open another document with a bigger picture, description, application etc etc.
(my workaround is to write a small text near the pict and then link it to the desired document)