Cloud storage support

DEVONthink
1

This is often repeated request, by many people, and each time it is met with dismissal.

I offer it again and ask you to just listen.

I do not have a large hard drive.
I do not use a NAS drive.
I do not use external drives.
I use cloud storage across different providers.

I need to catalogue and index collections across these different storage media.

I used to use DeEVONThink pro on my local and NAS drives to do this. I still have an up to date license for the latest version, but I haven’t been able to use DT for many years.

I am now moving back into the apple ecosystem and local first, but still with cloud storage, and I am having to work out how to replace DT pro within my tools set.

I am really sad.

Please stop telling us (former) users we are wrong and just accept this is a real need for many of us.

1 Like
2

We haven’t told people they’re wrong. We have clearly stated it’s technically infeasible and not data-safe to put your databases in a cloud-synced folder. Our concern and cautions are to keep peoples’ data safe, not to tell them what they want to hear and jeopardize their databases. And we are not the only application developers who issue such warnings to people.

PS: One of the things we see often are people using cloud services and having no appropriate local backups. If someone damages their database by putting it in the cloud, what recourse would they have?

7 Likes
3

So maybe change your model?

  • I understand why your approach and my needs are incompatible.
  • and please don’t start again with the lectures
4

Our format and model is mature and well-established for 20 years. In fact, many customers have databases well over a decade old. There is no simple “change your model” here, especially with terabytes (or more) of existing databases out there.

  • I understand why your approach and my needs are incompatible.

Not every application fits everyone exactly. There are many applications people use that “don’t make sense”, “don’t gain traction”, “don’t work they way they hope or expect”. If other options make better sense to people, we understand.

PS: There’s no lecturing involved. There is some clarification and restatement as we have to consider newer people who will come upon this thread and have no knowledge of the state of things.

7 Likes
5

@miloustale why not just leave your indexed folders in your cloud locations? I did that for years with no issues. Just don’t put the databases in the cloud.

Also, while I understand not wanting to go the whole NAS route, getting an external drive if you need it is pretty trivial nowadays.

6 Likes
6

It’s almost embarrassing how many I have amassed over the years. :open_mouth: :slight_smile:

3 Likes
7

Database local / files cloud - Yes indeed, I’d like that -DT doesn’t, and there are loads of threads on this

But please do correct me on this and instruct me

External drives are easy, but I don’t use them and haven’t done for 10 years or so. I do use cloud storage, however.

8

You can index folders in a cloud-synced location. However, it is best if the files are downloaded to the machine not only stored in the cloud. Most cloud services have settings to keep certain folders or files offline and local to the machine and that is ideal when indexing. That being said, if you are indexing files that aren’t downloaded, they will be downloaded when they’re accessed/viewed. There is no getting around that. And DEVONthink will not evict the file once you navigate away from the document. That is the responsibility of the user or the cloud service to manage.

1 Like
9

You and me both!

1 Like
10

Notice I say almost embarrassing to help justify the next one I purchase. “See, Jim? It’s really not embarrassing to have so many!” Hahaha :wink:

1 Like
11

I that been a long time since I tried this, but I do remember all sorts of problems arising - I had cleared my hard drive (onto external drives) to be able to index the files, then reset all the drives/folders/files to their desired online-offline states (of course, I get that to work on the file locally, it has to be local) but I kept having problems with DT re-triggering local downloads - this is pretty well covered in other threads. The model seems to be that all the files are indexed locally, and works best when those indexes are also maintained on local files, just giving the location to alternative locations in the cloud (conceptually rather than technically speaking-

  • but that simply isn’t my use-case and hasn’t been for years and unfortunately that isn’t entirely in my gift as I am working across different organisations and clients who are very much bigger than me.
  • -DAM tools that do work across different cloud providers are, as far as I know, cloud native.

How you reconcile that with the fantastic tool that I so enjoyed for years and sorely miss? Whilst I might like to hypothesise, that would be way beyond annoying, unhelpful and beyond scope.

Interestingly I don’t have the same issues with Bookends that faces a similar problem in a more limited scope.

12

I find this post a bit confusing. Instead of complaining vaguely, I think it would be more productive to describe your scenario – and ask for help on if & how DEVONthink could fit in that scenario.

From your original post it was not clear to me what your actual problem is, and it is still not entirely clear.

I feel some things are getting mixed up.

If the technical model/foundation is what allows DEVONthink to do all the amazing things it does, then this doesn’t make much sense. If a different model is used, it changes what features are possible to implement. It would likely no longer be the DEVONthink that we love. I’m not a software developer, and I don’t have the expertise to judge the technical choices made. But I trust the developers know what they are doing… And I appreciate their explanations – I don’t feel they are lecturing me.

Most of the time it is wiser to accept how things are, rather than setting yourself up for a host of negative emotions when reality does not conform to your wishes.

In general it is best to choose the right tool for the job. If the tool doesn’t fit the job, it is futile to blame the tool. Choose the right tool, or accept the limitations of the tool you’re using and work within/around them.

I mean:

– This is hardly the fault of DEVONthink.

What amounts of data are we talking about? Again, an actual description of the scenario/use-case is necessary to really give input. I don’t understand why you can’t just import/index the files you need – I assume you don’t need to include the full contents of your clients’ cloud storage?

I don’t know this space very well. What DAM tools are you thinking of? And if these tools fit the job better, why not use them?

While you can use DEVONthink to catalogue media in some ways, it’s not really a DAM.

What is different in your experience with Bookends? (I suspect the more limited scope is a key factor.)

4 Likes
13

Without knowing more… Have you considered NeoFinder? (formerly CDFinder)

I can’t tell you much about it, as I haven’t used it – but if you’re looking for something more in the DAM category, I think it’s worth checking out. I have looked at their website now and then when considering workflows and set-ups involving many data locations.

It looks particularly useful for audiovisual data, but I think it would be useful for many other things. Basically, any large amount of data spread over multiple locations where you want to keep a central index/catalogue.

But this index probably takes up some space, so if a small internal hard drive is a factor you won’t/can’t change, I don’t know if it would be any better. I think that is a limiting factor in most cases.

14

I purchased my Mac Mini with the minimum base storage
knowing an external drive is an easy expansion option (4TB portable HDD)

I use cloud storage for my DT sync datastore (all databases)
however each device needs a local copy of specific databases

1 Like
15

Well, why not?

Why, on the one hand, is cloud storage so valuable to you that you’re willing to ignore the abundant evidence (not just from DT) of risks associated with cloud storage? While, on the other hand, you are insisting that DT should ignore those risks and change their design even though there are safe, inexpensive, alternatives?

6 Likes
16

You are right - but the post wasn’t meant as a critique of DT, but as a statement of need.

Why don’t I use xyz and why is that DT’s ‘fault’ - that’s not in scope.

But the statement could be much better, so you still right with no ‘buts’!

17

Debate not in scope - the circumstances are simply that there can be no NAS.

In my particular case because it is a physical item that has to be attached to a specific network, which may be accessible today, but not tomorrow, whilst I still want access to my files.

Think nomads.

18

Use-model.
1 - Don’t have the place for physical media / move around etc… (yes, I do know external HDD are portable and may have to reconsider)

2 - distributed files across teams etc

3 - different client limitations

19

I’m laptop based, not located.

The sync data store

  • enables you to have one instance running on one machine whilst another is turned off?
  • is about as big as the original databases that you are syncing?
20

Zotero’s documentation includes a good explainer of why you shouldn’t store the data directory in cloud storage:

Database-backed programs like Zotero rely on file locking to ensure file integrity, but cloud storage systems generally don’t honor such locks. If you wake up your computer with Zotero running, and then Dropbox pulls down a change from another linked computer and updates part of the database file, Zotero will be unaware of the change and will corrupt the database when it next writes to the file. Even if a cloud storage tool did honor file locks, you would simply end up with two conflicted copies that were impossible to merge, and those conflicted copies would quickly proliferate in the Zotero data directory.

1 Like