Import remote encrypted DB

Hi,

I’ve created an encrypted database on a laptop with around 8000 items, 70 gb total, and uploaded it to my sync store. Sync was successful.

On my desktop, I then tried to import it from the sync store (right-click, import encrypted database).

Three strange things happened:

  • DT asked me for a database name and the password twice (password + verification, as if I was creating rather than accessing the database).
  • It started importing 25000 elements, although I have no idea why the count is so high.
  • I asked DT to save the database on a network volume, and it only saved a sparseimage of around 250 mb before proceeding to fill up my entire internal SSD and generating an error. I couldn’t even find the location of the temporary files. Space was only freed when deleting the database from within DT (on the desktop).

DB is OK on laptop and sync store. What do I need to do in order to import it elsewhere?

Thanks.

  • DT asked me for a database name and the password twice (password + verification, as if I was creating rather than accessing the database).

And what is set in File > Database Properties on the Mac you generated the database on?
This was just discussed in the Jan. 30 blog post: Understanding Database Credentials

  • It started importing 25000 elements, although I have no idea why the count is so high.

As has been discussed many times before, the number of items being synced is not the number of documents in a database. It is not a file copying mechanism. It chunks files for transport as needed and also has transactional data involved.

  • I asked DT to save the database on a network volume

Are you hardwired on a gigabit Ethernet connection? If not, it is not recommended to put DEVONthink databases on a networked volume.

and it only saved a sparseimage of around 250 mb before proceeding to fill up my entire internal SSD and generating an error.

What error?

1 Like

Above is incongruent. “Uploading” is not “syncing”. Frankly, not sure based on this what you did.

I have not set a database user and password. I only added an encryption password to the database when creating it. What I found surprising was the fact that, when trying to import it from the sync store, I got the same window as when creating a new database, asking for the password twice.

I missed that, thanks.

Yes, I am.

In the DT log, “no more disk space” or something to that effect. Only then did I discover that my free disk space had been reduced to 10 mb. Like I wrote, I couldn’t find the temp file and the disk space was released only after deleting the half-imported database from inside DT.

Does DT need the equivalent space on the internal SSD when importing, even if creating the database on a networked volume? Even if so, it should not eat up all remaining space in the process without warning first, or a at least checking regularly that at least 10-20gb are free.

Although a successful sync was indeed performed, I used the term “uploading” instead of “syncing” because this was the first creation of a copy of a local database on a remote sync store. I sincerely hope you are not offended.

I am not offended.

So, then after you synced, you then tried to import encrypted database from a “syncstore” which is not a database. Files that exist on a sync location are not databases and I’m surprised it could be “imported”.

Maybe all this is just mis-use of nomenclature. Call me confused, not offended.

The database is visible in the “remote” section of the sync store I’m trying to synchronise with. A local copy has not yet been created. Isn’t a right-click + “import database” the correct way to sync it to a local volume? Whatever I do (import database or import encrypted database), DT fills up my system SSD instead of saving the database to the chosen location.

Oh, I now understand what you are saying. I thought you meant when looking at “sync” store, you were looking at the remote location with as a file…but you are in the Menu: Settings → Sync tab.

I have not set a database user and password. I only added an encryption password to the database when creating it.

Please clarify what you are referring to here.

DT fills up my system SSD instead of saving the database to the chosen location.

Again, DEVONthink is not doing simple file copying. It’s not just passing data from point A to point B. It is receiving raw sync data locally, data it has to qualify, before passing it into the database. That raw data is going to be stored in DEVONthink’s Application Support folder while it’s processing. And your home directory is likely on the internal boot drive – and ideally should stay there IMHO.

Database was created with File > New Encrypted Database. The password was entered twice in the required fields. The local file has a .dtSparse extension.

When I right-click on the database and select Database Properties, there is nothing in the User and Password fields. I never set them to anything.

Does this clarify my setup?

I understand, this seems to confirm what I was suggesting above, i.e. DT needs, for the first sync at least, free space on the boot drive of a size at least equal to the size of the database being synced, regardless of the space available on the volume where it was asked to be saved locally (i.e. an external drive or network volume).

Does this clarify my setup?

Indeed. We still have people using the database credentials who think it’s locking or encrypting their databases somehow.

I understand, this seems to confirm what I was suggesting above, i.e. DT needs, for the first sync at least, free space on the boot drive of a size at least equal to the size of the database being synced, regardless of the space available on the volume where it was asked to be saved locally (i.e. an external drive or network volume).

I’d say yes, that’s expected.
How much free space do you have? And a 70GB is not a small database for sure, unless it’s full of media files.

I don’t have 70gb free on my boot drive, and the database is indeed just a (big) bunch of personal photos that I’d like to access from laptop, desktop and DTTG. I know DT wasn’t designed for that and I’ve looked at several selfhosted photo galleries, but I’m reluctant to use them because of security reasons and because they don’t offer the flexibility and familiarity I have with DT. I’ll try and figure something out. At least the problem I have is clear now. Thanks.

Do you need simultaneous access to view the photos on all the devices?

Not really, I just need a central server that gets updated whenever I modify the files on a device and propagates the changes to other devices. DT seemed to fit the bill nicely because I’m used to it. Tried other solutions (nextcloud, seafile) and found them less useful for this purpose for different reasons.

Well DEVONthink To Go doesn’t support indexed files so you’d have to store the entire contents on your mobile device or try to use a shallow sync and download contents on demand. Unless they were critical photos, I would think either option would be less than optimal.

I agree, I’m still looking for a solution, optimal or even half-decent :slight_smile: Downloading on demand on DTTG wasn’t a problem as I would only use it for viewing them, most work would be done on laptop/desktop.

So you want remote access as well, e.g., on your laptop?

Yes. Download on demand on mobile is OK. I essentially need DT but for photos… although I know this question has been raised in this forum several times. I know DT wasn’t designed for this, but if there wasn’t the issue of boot drive temp files I would have worked something out with my current setup.