Accessing DT from several machines, advice? (noob level)

DEVONthink Tips
1

I have a situation that I think I should probably undo and rework, but I would like advice, if anyone has time. I am not a power user and need plain-English explanations.

Here is the background. (There are probably going to be things in this description that make power-users wince. Sorry.)

  • I am a writer and work both on a main iMac and also on a laptop when I am on the road. I have DT installed on both.
  • I keep most of my life in Dropbox.
  • I have my DTP databases in folders within my Dropbox.
  • I never have DT open on the two machines at the same time (well, once or twice by accident, but not as a working rule).
  • This was all set up pre the arrival of Sync.

I’ve picked up from discussions in these forums that this may be a bad set-up. So, what would the advice be to change it? Do I have to:

  • move DT out of Dropbox on my “main” computer
  • which would make it not available to the road computer?
  • How then do I make the DT databases accessible to the second computer?

Thank you for any thoughts.

2

Hi. See the “Synchronizing” tutorial for 2012/12/13. Does this answer your questions?
devontechnologies.com/suppor … rials.html

3

Thank you for pointing this out, but it doesn’t actually answer my question.

4

After some noodling, I discovered this thread:

which appears to answer my questions. I will undertake the process described in it, maybe creating some small non-critical databases to test what I am doing. However:

If you read through that thread, it’s really evident that the OP is as confused as I am (in fact, I’m probably more confused) and that things have to be explained multiple times because documentation doesn’t exist and the tutorials aren’t helpful.

This is such a persistent problem in DT, that whatever explanations exist are just not pitched in a way that is helpful to all levels of users. I know of so many journalists who have simply given up on the program because of this. It’s discouraging.

5

Hmmm… It seems to me that the video tutorial did cover all of the bases. But, maybe I am the one that is confused about what problems you are encountering.

Thanks for the link to the thread. I hope it will help folks. I think the key thing to understand out of all of this is that the database itself should not go into Dropbox. It makes sense, but does work differently than other programs.

My recommendation for the DT folks would be to streamline the steps and the interface, if possible, and then the documentation / videos will be easier to follow. Certainly, I’ve heard from lots of people that the learning curve in DT is pretty steep compared to other apps.

6

You absolutely must not store your databases in DropBox. The most failsafe way to move your databases between computers is to use a portable external hdd or flash drive. Some might see that as a very 20th Century solution, but it is by far the most robust. You simply save your databases to the flashdrive (using a suitable back up plan, of course), and then plug it into your other computer to have your databases available there.

7

I agree with Kinsey’s remarks. Do not store or run DEVONthink databases within Dropbox or other cloud host, as they were not designed to handle dynamic databases.

And that “20th century” method of transferring a database from one computer to another by copying it to a portable drive or flash drive, thence to the second computer is indeed simple, straightforward and robust. I did that for years before the advent of the Sync procedure, and never had a problem.

The next generation of DEVONthink and of its Sync procedure will seem more intuitive if one needs to share databases among multiple Macs, including a workgroup. But I’m sure there will still be occasions when I’m on travel and will lighten the load by leaving my Mac at home, only carrying my iPhone and a portable drive that holds my databases. When I reach my destination at a friend’s house (he also works for DEVONtechnologies), I mount the portable drive on one of his Macs and I’m all set. (My databases are self-contained, so this is a simple process.)

8

Maryn, it would be wonderful if DEVONthink databases could be shared and synced among multiple Macs simply by storing the database files in Dropbox, or Box or any of the other cloud services on the Internet.

Unfortunately, there are two problems that emerge.

  1. DEVONthink databases are single-access databases. Simultaneous access to a database by more than one DEVONthink application can cause confusion and database damage. If stored in a cloud host, failure to close the database before it is accessed by a different Mac can cause problems, and it’s all too easy for users to get into trouble with this approach.

  2. Even more seriously, DEVONthink uses Apple’s file system to manage databases and their contents, but cloud hosts such as Dropbox, Box and others do NOT use Apple’s file system. Each of them differs in important ways, such as the ways package files are handled, limits on the size of some files, and assumptions that stored files are static (but DEVONthink databases are dynamic in file storage, as files can often be moved inside the database for performance reasons). Dropbox works very well as a simple file copier of “simple” files such as word processor files, but becomes problematic when faced with a DEVONthink database.

The Sync procedure of DEVONthink offers the ability to share and synchronize databases among multiple Macs. Technically, the most straightforward way is by direct connection of two Macs. But that method doesn’t meet the needs of many users, especially when the Macs are in different locations.

To enable Sync of Macs in different locations, the concept of a Sync “store” is introduced. The document and metadata files contained in a database are copied to the “store”, which may be held on a portable drive or in Dropbox. (I won’t talk about WebDAV here, which has complexities such as the ways in which authentication and other issues are handled by different hosts.)

The beauty of the Sync “store” approach is that it bypasses the two big problems noted above. Simultaneous access of a database is avoided, and the “store” doesn’t contain the big database package file itself, and the “store” files are static “simple” files, not dynamic in nature. A different Mac can access the “store” via Sync and create or update a shared database.

Many of our users employ Dropbox as the vehicle that holds the Sync “store” and allows Sync via the cloud. The tutorial video and the user documentation PDF describe the steps to set up Sync. Note that Sync doesn’t use the standard access provided by Dropbox, but communicates directly with Dropbox. For that reason, and to prevent unnecessary duplication of storage space, directions are provided to remove certain files from one’s Dropbox folder.

Does Sync work with other cloud hosts, such as Box? Currently, Sync is “tuned” for Dropbox, and uses some APIs that are specific to Dropbox. It appears that each cloud host differs in some respects. That makes it difficult to develop Sync for the range of cloud hosts. Dropbox is currently supported because it is popular. (In my case, as I live in a log cabin in the woods and have satellite Internet access, the size of my databases, low upload and download speeds and a monthly limitation of 15 GB during working hours eliminate cloud Sync.)

A limitation of the current Sync procedure is that the user of a shared database becomes aware of changes to the Sync “store” by another Mac only after invoking a new Sync procedure. A future generation of Sync will likely be able to send database changes almost in real time.

The simplest case of Sync is when the user makes database changes on one Mac, then moves to a second Mac and updates it to the changes made on the first Mac.

Sync gets more complicated when changes have been made to a database on more than one Mac, and Sync is invoked. For example, suppose a word processing document has been edited differently on two Macs. This raises the possibility of conflicts and how to resolve conflicts.

In the case of a shared database accessed by a workgroup, it would be prudent to make someone the manager of the database, who would institute rules for use of the database. Otherwise, Sync conflicts become more likely. Think of a rogue workgroup member, who decides to reorganize the database to fit his own wishes (worse yet, two rogues who disagree with each other). :slight_smile:

Do not regard Sync as a comprehensive backup system. Do use a real backup system, such as Time Machine or other backup utility, or Database Archives.

9

Thank you all for your comments overnight. I do want to point out that I have gotten the message, loud and clear, that my databases should not be in Dropbox. I am not arguing with that assessment, I am asking for help in understanding how to move them out, and then set up Sync on a second computer, given that the 2nd computer previously had access via Dropbox. I expect there to be some stutters.

I would like to point out: I have been a DT user for years, and never, except in the forums, encountered any instruction NOT to put the databases in Dropbox. Why is that not made clear? It ought to be simple to add to the manual user instructions. For instance: The first time I used Sente, a bibliography program which also has sync built in, it popped up a warning that said, basically: “Hey! We can tell you’re putting this in Dropbox, don’t do that!”

The number of people on these forums who express surprise that their databases ought not to be in Dropbox seems to me to indicate that such a warning would be beneficial.

10

And Bill, thank you for your clear explanation of the underlying rationale, I appreciate it.

(And yes, I redundantly back-up, Time Machine onsite and Crashplan offsite. I don’t really consider Dropbox “back-up,” except in a secondary way.)

11

Moving DEVONthink databases out of the Dropbox folder is usually a simple matter.

First, Quit the DEVONthink application, as an open database should never be moved. Move the database to another location on your computer, probably your Documents folder.

Launch the DEVONthink application. Choose File > Open Recent, then the Clear Menu option to erase the previous location information. Open your database(s), either by double-clicking them in the Finder, or via File > Open Database….

Check your database to make certain that the Paths to any Indexed content are still valid. If so, the move worked without incident.

Finally, run Tools > Verify & Repair on your databases to check for possible errors. If errors are found, run Tools > Rebuild Database; afterwards, check the Log (Window > Log) for a list of any files that were not included in the rebuild (the list can be saved for future reference).

12

Hello, I am picking this thread back up again to ask some further questions. I would like to describe a situation I have encountered and ask for advice.

To begin moving my databases out of Dropbox and begin using Sync, I first created a small new database, in my Documents folder, and set it up via the Sync video tutorial. This worked without incident, both up to Dropbox and down to my laptop.

I then copied (Mac user: Finder> File> Duplicate) the smallest of my existing databases, moved the copy out of Dropbox into Documents, gave the copy a new name, used File> Open recent> Clean menu to clear location information, selected the new database, and opened it — but it opened with the name it formerly had. For clarity: It was originally Foodfodder; the copy was first Foodfodder copy and then Food1. I selected Food1 to open. When it opened, it displayed the name Foodfodder.

The paths to the documents within the database are still good, so it seems to have come over intact, but why it is switching back to its old name, and is this a danger signal?

I’d like to get this sorted out before I try transferring a larger database out of Dropbox, so I would appreciate any thoughts.

(ETA: I tried this a second time to make sure this was happening consistently, and it is. Chose a database, Duplicated it in Finder, renamed the copy, moved it to Documents; opened DTP, Cleaned Menu, navigated to the copied version in its new location, Opened it in DT — and it opened under its former name. FWIW)

13

So, korm, you are saying I should: Create a new empty database, open it in DTP, open the existing one (the one that resides in Dropbox which is a Bad Thing) in DTP, and then drag stuff from the existing to the new? Just trying to be sure I understand.

I don’t understand how this accords with Arnow’s advice:
[url]Syncing and database location/storage]
to copy a database, sync the copy, and not delete the original until you know the copy has synced properly. if I drag things from the original, won’t I be emptying out the original?

14

The OP asked me to comment.

The most important step is to remove the DEVONthink databases themselves from the Dropbox folder. I would treat this quite simply, by the following steps:

On each Mac:

  1. Quit the DEVONthink application. (Avoid copying open databases.)

  2. In the Finder, Command-click on the DEVONthink databases (their filename extension is .dtBase2) to select all of them. (Command-click allows non-contiguous selection of multiple files.)

  3. Hold down the Option key while dragging the selected files to copy them to a different location, the Documents folder. (DO NOT YET DELETE the database files from Dropbox!)

  4. Launch the DEVONthink application. If the databases still located in Dropbox were automatically opened, close them.

  5. In DEVONthink, choose File > Open Recent > Clear Menu. (This will clear DEVONthink’s memory of the previous location of the databases.) Now open each of the databases copied from the Dropbox location to the Documents location, so that DEVONthink knows their new location.

Repeat those 5 steps on each Mac that holds copies of the database in Dropbox and that are to be used for database work.

Afterwards, select all of the database files (their filename extension is .dtBase2) that are stored within the Dropbox folder, Control-click and select the option to move them to the Trash.

Run Tools > Verify & Repair on your databases on each Mac to check for any errors, and run Tools > Rebuild Database if errors are reported. If Orphan files are shown in an Orphans group, refile them into your database organization.

Now your databases are safely stored for use and you will be able to Sync them among your Macs.

Although multiple steps are involved, they are standard OS X procedures for copying files and can be quickly performed on each Mac.

15

Addendum: As I work primarily on a MacBook Pro and cannot count on always having Internet access, I don’t have any scheduled Sync activity, but only use Sync Now.

If you have scheduled Sync, including sync on launch, quit, hourly, etc. I should have mentioned the advice of others:

Stop activity on the Mac for 15 minutes, then tell Sync to pause now. Then follow the steps of my above post.

16

Hello, I am revisiting this thread because I am having difficulties moving my databases out of Dropbox, and am seeking advice.

I copied a database, moved the copy out of Dropbox to my Documents folder, opened it, changed the database’s name both in Finder and in Properties, cleared the menu of Locations, reopened it, and checked out a number of entries. Everything seems to be there. However. when I attempt Sync, I am persistently getting “Verification failed. Please repair the database.” When I use Verify and Repair, it reports 21 persistent errors that seem to be blocking Sync.

How can I identify where these errors are and clear them (if that is what I should be doing), so that I can Sync this database and thus access it from multiple machines? Thank you.

17

Errors that are identified by Verify & Repair should be displayed in your Log. If the Log is not automatically displayed following Verify & Repair, use Window->Log to call it up. Log will help you to identify the errors. If you index a lot of files, some of your ‘repairs’ will probably consist of fixing broken index paths.

I can’t advise you how to resolve your Sync issues because I avoid using the procedure. Based on my initial attempts to use it, it is temperamental and overly complicated.

18

When database errors are detected, choose Tools > Rebuild Database. After completion of that procedure, choose Window > Log and examine the Log for a list of any files that failed inclusion in the rebuilt database. The Log list can be saved for future reference.

Orphan files are a special case of database errors. When files are captured into or created within a database, they are stored within a folder inside the database named Files.noindex (in the case of Indexed files, the documents are stored outside the database). Documents in the database are each assigned a Path to its corresponding file.

If the Path to a document becomes lost or damaged, that document can no longer be displayed, printed or edited—a missing file message is displayed when the document is selected. When Tools > Verify & Repair is run, files within the Files.noindex folder for which no valid Paths exist are moved out to a special group, Orphan files.

If an Orphans group exists and has contents, those contents require action by the user. Here’s the procedure in DEVONthink’s Help to take action for DEVONthink Pro or Pro Office databases:

"Orphaned files: Orphaned files that could not be imported while repairing are logged and moved to the folder “Orphans” inside the database package. To access the contents of the package, show the database package in the Finder, right-click it, and choose Show package contents from the contextual menu. Move the files in the “Orphans” subfolder to a save location, e.g., the Desktop, or directly re-import the files by dragging them to DEVONthink Pro Office’s Dock icon. Please do not change anything here except for rescuing the orphaned files.”

For users of DEVONthink Personal, the database is a folder named DEVONthink 2, and doesn’t require the “Show package contents” step. Simply select the folder to open it.

Tip: Since Lion, Apple has hidden the user Library folder. But the DEVONthink Personal database is located at ~/Library/Application Support/ and the Global Inbox for DEVONthink Pro and Pro Office users is located at ~/Library/Application Support/DEVONthink Pro 2/. To make the user Library folder visible so that one can navigate to the desired file, in the Finder choose the Go menu, then Go to Folder. In the query field, enter “~/Library” (without quotation marks). There it is!.

19

So say we all.
But thank you for the comment.