Managing large DT databases

Hello,

I have been using DT for a few years now but in conjunction with a few other tools/databases. My research partner and I are keen in centralizing all our data in our DT database that we sync between the two of us and multiple installs (I have purchased 3 keys, will probably need one more for a research assistant). We sync using dropbox and it has been working well but our database at the moment is mostly text files with a few pdfs of books and articles (we are historians). The next step for us would be to store the photos we take of documents at the archives in it. Right now those photos are in a shared dropbox folder. We have thousands and thousands of images. I am wondering if moving these images to Devonthink might slow down our syncs or have any impact on usabilityin general. We dont HAVE to have them on DT, I suppose. We could import just the images we actively use or need to annotate and keep the deparate Dropbox folder as an archive of all the unedited photos.

Just wondering if anyone else has similar sceneario….

Alexandra

1 Like

How do you manage Dropbox on the individual computers of the partners? Does each partner sync all of Dropbox to their computer, or do they only download the files they need? If the former, then you are in essence already syncing the images to every partner’s computer. Hard to say without specifics in terms of number of images and total blob size what the load on DEVONthink sync would be if you imported them into the shared working database.

However, I think you answered the question about merging the images into your shared DEVONthink database when you wrote “We don’t HAVE to have them on DT”. If there’s no resounding Yes, then don’t do it.

Unless the partners are all working with the same images at the same time, I would suggest each partner have a private working database of their own where they can index the images from Dropbox that they need at the time. Not all; just what’s needed.

The private databases can reside perfectly alongside the shared database inside DEVONthink, which is really terrific at supporting work across multiple databases simultaneously.

If an image or group of images becomes necessary to have in the share database, then import the image(s) on an as-needed basis. Keep it simple.

3 Likes

Welcome @guerson
@korm’s response is a good one.

I am wondering if moving these images to Devonthink might slow down our syncs or have any impact on usability in general.

They would slow down sync on the initial upload and subsequent import to the other Macs, but there’s nothing inherently slow about having images in a database. Their usefulness in DEVONthink depends on how you would work with them practically.

2 Likes

The question is also what additional benefits you would get from bringing all these images into DT. You might, for instance, want to use linking to them directly from within DT, and that’s a good use case. You could also replicate select image sets from a single-truth repository to other folders that are dedicated to, say, a single research output, a grant application, whatever. That’s another good use case where DT would shine.

In my own research use (somewhere between literature, history, book history, and more!), however, I have a large database of images (mostly of manuscripts and rare books), about 65 GB. But because I never really invested in creating good metadata and most images have the camera’s default image numbering for a file name, it’s the one database that I use where DT offers me very little benefit over, say, Finder or a shared Dropbox folder. I never search it and can only get to what I need by navigating folder hierarchy (Library / Collection / Shelfmark). It suits my needs, but isn’t often used in the ways my other databases get used.

Once more, a lot depends on the nature of your materials. If your archival materials contains, e.g. typescripts you can OCR the lot and unlock DT’s powerful search and AI. In my case, I am mostly collecting images of medieval and renaissance handwritten documents and there is, as yet, no reliable automated paleography solution!

Welcome to the forum, and good luck!

3 Likes

ooooh, I had not thought of working across multiple databases but that’s a good idea. I will have to investigate the workflow. Thanks!

1 Like

Our case is like yours. We are medievalists, which means, no, nothing can be OCRd, unfortunately. We do take brief notes on each document we photograph before we photograph it so that we have a handy searchable repository but like in your case, locating a particular image does require us to browse to a particular folder hierarchy.

Would love to chat with a fellow historian on how they use their DT database for research, if you wouldn’t mind…

Are you familiar with our friends over at https://www.devonthinkforhistorians.com ?

1 Like

Sure, why not. I will drop you a private message to follow up. Happy to chat DT with a fellow medievalist.

yes! Need to see what’s new over there as I have not viisted in a while.

1 Like