ER: Adding Cloud Storage Providers (My thoughts)

This is largely going to be my order of preference and sample use cases/examples of why I care. Could probably be consolidated with other posts or something I don’t mind if it gets moved or whacked or something.

  • iCloud — because I trust it and it’s reliable for me now (but the switch to Sierra will be rocky I suspect).

  • Google Drive — because enterprise/highered users that use GoogleApps will have it and it’s pretty good (Dropbox handles metadata better)

  • OneDrive — I have no idea what the level of effort here would be, but again for enterprise/highered that uses Office365 et al this would be great (and I’d have a reason to use my OneDrive for something other than copies of my presentations and assessments)

  • AmazonDrive — this one is interesting because it’s very inexpensive and a lot of people have access to it for free/free-ish (I bought a USB harddrive a while back and they gave me a free year) so I have been using it with Arq and it has been reliable but the checkin/checkout model for end users isn’t great so an application like DEVONthink would make it a much more interesting service option (admittedly only for users of DEVONthink)

Finally,

  • AWS S3, B2 Block Storage, Google Storage, etc — I think really this is the best of all of them but the user adoption would be very difficult. It isn’t trivial to configure IAM and policies and all the required key stuff and most people would be turned off by it, but the price is so low it’d be very handy for people with large datasets!

I think Google Drive sync should be a priority for a simple reason:

  • A substantial number of DT users are students or academics, and a large number of these universities offer their students Google’s education app system which includes… unlimited Google Drive storage. As such, not only is it quite useful, it is also a good fit for Devonthink, for another simple reason:
  • Devonthink seems to encourage an import rather than an indexed workflow. I.e. you essentially copy your files into DT’s database. If you want to index your files instead, DT does not work that well. The syncing of indexed folders is somewhat slow and, particularly damaging, it is a one-way sync. Files added to an indexed folder in DT do not get synced back into the file system - at least not automatically. This means that DT works better with an import strategy and, in turn, this means that to sync with Google Drive, at the moment you would have to do a sync to the local file system, wasting a lot of hard drive space in the process (you are duplicating your files - one copy in the DT database, and another in a Google Drive folder). There is no good alternative for this workflow at the moment.

Just curious – do you actually need 9 more locations for sync stores in addition to the ones already provided?

I do not, but I can easily imagine a scenario where someone else would not want to use Dropbox Professional when they already have a terabyte of storage available on Amazon Cloud Drive or OneDrive from their employer. I wouldn’t use nine, I’d only use a couple. The only reason I pay for Dropbox is because I can’t use AWS S3, Amazon Cloud Drive, or OneDrive. Some users in Higher Ed have OneDrive, others may use Google Apps — why not lower friction and permit users to use their service provider instead of asking them to subscribe to another one they don’t want to use?

Cloud services are plenty and they often come and go quickly. Also, there are APIs specific to each. Having to code for and maintain code for each service is not a trivial task, nor is it ultimately a good use of resources in the long term. And note these services change their APIs, which leads to breakdown in Sync. This leads to a flurry of Tickets that “Sync DOESN’T WORK!!!” when the breakdown is from their changes, not ours.

WebDAV is a good catch-all for other providers we don’t specifically support. You can always contact your desired provider and request they offer WebDAV services.

Sure, why use something with modern features, fast performance and reliability when you can use something that is well-documented, slow, and unable to implement frivolities like metadata? :stuck_out_tongue:

Reliability is established over the course of time.
“modern features” are transient. New “modern features” come out all the time.
“Fast performance” is a moving target and uncontrollable. As they say, YMMV.
:mrgreen:

WebDAV is definitely not slow (sometimes even faster than the “native” API) but the Finder’s support for WebDAV is definitely horrible. In addition, the synchronisation doesn’t and can’t depend on the metadata support of services & servers and therefore this doesn’t matter.

OT, It may come as a surprise to most of the software developers, but “Sync DOESN’T WORK!!!” can easily be avoided by implementing clear and detailed error messages :slight_smile:

Not sure what this means, or why it is relevant to this thread.

Just to add to this conversation from my own use cases in higher ed as a social scientist. I often need to use different sync stores because of data location and privacy law. Dropbox is great for a host of reasons, but they don’t allow users to specify the country in which servers are located which is crucial for data which is under some protected protocol. WebDAV works ok here, but Universities are surprisingly slow to implement open standards-based solutions such as this and often prioritise services which can be bought efficiently from a third party, like OneDrive or Powerfolder, etc. (Which do ensure European data locations) And further, providers which emphasise security (with features like zero-knowledge pre-upload encryption) and privacy are often smaller players, but have APIs which aren’t fundamentally that different from the big players (box, Dropbox, OneDrive, etc.)

Some of you may chime in to note that some of these, like OneDrive or powerfolder are open to WebDAV access, which is true, but devonthink often chokes on WebDAV providers that require captive portal authentication, rendering 403 errors, whereas sync which uses a could provider’s api seems to be a bit less idiosyncratic, which to be fair to WebDAV is exactly what one should expect with an open standard with a range of implementations.

Just my two bits…

This is not a super helpful answer. Services that are provided by companies like Amazon (S3) and Microsoft (OneDrive) are likely in it for the long haul and have huge user bases. (My opinion is that they will outlast Dropbox.) It seems to me that it is Devon’s responsibility, having created a good syncing scheme, to support major industry products, even though it may require development resources. As others have said here, different customers have different remote storage needs that are based on their own company’s policies, geography, financial decisions.

We try to be responsive to user needs and requests. But we are a small company. As BLUEFROG noted, it isn’t a trivial matter to code Sync for a host such as Dropbox, which was done because it was the most used and requested by our user base when Sync was developed. The issue is compounded because every cloud host has its own set of APIs, which they may change over time. Every API change leads to broken Sync (yes, Dropbox has done that), requiring diversion of developer resources from our own priorities, to fix the issue in a new maintenance release.

That isn’t to say that we may not change or expand the list of supported cloud hosts in response to user community needs. But were we to regard it as our “responsibility” to support all major cloud hosts in Sync, we would incur significant added development and support costs, requiring a higher cost of DEVONthink for our users. That’s reality.

That’s why Sync supports WebDAV. Although there can be issues with the way some cloud hosts support WebDAV (such as authentication issues), WebDAV does offer at least some users an immediate expansion of choice among cloud hosts. The issues of speed and metadata transfer are not as serious as assumed in one of the posts above. Sync does handle document metadata in the WebDAV mode.