Easy 3rd Party Integration Solution!

DEVONthink To Go Feedback
1

The ability to work entirely using an iPhone or iPad (free from the shackles of the desktop/laptop) is incredibly appealing to me and many people (and to all the Windows PC users who have iPhones and can’t even use DEVONthink Pro Office for Desktop).

However, as the previous “Scanner Pro Integration” thread revealed, there’s currently a super clunky 9-step process (for EACH scan you create) to share scans with DEVONthink To Go, which is a real buzzkill when you have to scan lots of documents every day:

The above sounds like a horror story but the DEVONthink To Go 2 integration is actually equally bad in ALL other 3rd party apps that create documents. There’s always just the slow, generic “share” panel which takes a bunch of clunky steps to get to and a bunch of clunky steps to finish. (And never preserves the original filenames.)

However, this gives me an idea… and idea that would improve all 3rd party app integration, for free, without talking to any other software developers, and with minimal coding effort from DEVONtechnologies.

  • Make DEVONthink To Go 2 (DttG2) use the Dropbox API (and all other supported cloud storages, but I’ll use Dropbox in this example).
  • Add an option for DttG 2 to WATCH one or more specific NON-DATABASE Dropbox folders, such as “/Dropbox/Scanner Pro”.
  • Provide options for what to do when DttG 2 detects a new file in one of the watched folders, such as “Move into Global Inbox database and delete original file” or “Copy into Global Inbox database and keep original file”.
  • Make DttG 2 check all watched folders via the Dropbox API at startup and every time it detects that you switch back to DttG 2 from another app, and at periodic intervals, to ensure that DttG 2 always quickly detects the files as they come in, for smoother integration.
  • With this solution, there’s NO NEED to integrate with ANY 3rd party apps (scanners or any other apps!). Instead, you simply let the 3rd party apps write their files to a specific Dropbox folder, and then DttG 2 detects the files in that folder and copies/moves them into your database. This preserves filenames, timestamps, and is ten billion times faster and easier than the clunky “Share” panel process in 3rd party apps.

So with this kind of integration, an example setup for Scanner Pro would be:

  1. Scanner Pro setup: Create a Workflow, set it up as you like it (such as "Name the file “Receipt YYYY-MM-DD.pdf”, “Email it to my assistant”, and “Place it in /Dropbox/Scanner Pro”, and “Delete the original scan file from Scanner Pro, to clean up Scanner Pro afterwards”).
  2. DEVONthink To Go 2 setup: Set it to watch “/Dropbox/Scanner Pro” for new files, and choose “Move file to Global Inbox and delete original from watched folder”.
  3. Usage: Scan a document in Scanner Pro, tap the “Archive Receipts” workflow, and you’re done (the file will be on dropbox). Switch over to the DEVONthink To Go 2 app and it will detect the new document and download it into the database.

It means that all apps (and there are lots) that support saving files to Dropbox can be made to easily integrate with DttG 2 without needing to individually share (and rename!) files one by one via the clunky Share process. It’s all just transferred quickly and easily between the apps, via the cloud, with no need for the user to interact BETWEEN the apps. It happens automatically.

This would enable all kinds of cool workflows. Windows users with an iPhone/iPad (who have no ability to run a desktop version of DEVONthink) are plentiful, and they could make DEVONthink To Go 2 for iOS watch a folder like “/Dropbox/Send to DT”, and then just dump files in there from their computer, to instantly get them into their document repository. Cloud folder watching would open all kinds of wonderful possibilities and freedom and interoperability between all iOS apps and operating systems that simply support cloud sharing. Another example would be having a shared Dropbox folder with a colleague, and letting DEVONthink to Go 2 for iOS index all files your colleague adds to the folder. Cool huh? The possibilities are endless!

2

I think if DTTG was a document provider it would solve your issues. I wouldn’t want my databases syncing with two places at the same time – especially since Dropbox is part of the Borg. Hey – is there even an app that can sync in the background with Dropbox?

“All”? Really?

Say, DEVONthink aside – why not tell us what app on iOS is your benchmark, wow-that-cool, design for an iOS share sheet? It really helps to know what the gold standard is so maybe the developers have an exact example to look at.

3

DEVONthink To Go 2’s Share panel:

  • Doesn’t preserve original filenames. Automatically generates “File: 2016-10-08 13:52:12.pdf”-ish filenames.
  • Must select target database to save file to every time.
  • Can only pick/move/copy a single file at a time. Never multiple files.
  • Must be done manually. Cannot be automated via workflows in any apps, since the OS handles the Share panel and the source/destination work.

DEVONthink To Go 2 as a Document Provider:

  • Preserves original filenames.
  • Lets you navigate directly to files within application bundles. The OS is what generates the document chooser interface, and doesn’t support special handling of files (no special app code can run), it’s really just a “display all of the files in this physical folder” kind of API. Therefore it’s likely never going to be compatible with DEVONthink To Go 2 since DEVONthink databases use their own convoluted “Index” folder with lots of randomly named subfolders that internally arranges all files in your database. Letting users navigate that internal database folder-mess would be a horrible experience (“Documents.dtBase2/Files.noindex/pdf/3/2008-01-01 Some document.pdf”, so intuitive eh?), and would risk destroying the entire DEVONthink database. For that reason I doubt that Document Provider functionality will ever be added. They’d first have to restructure how DEVONthink databases organize their file-folder structures.
  • Document Providers only pick/move/copy a single file at a time.
  • Must be done manually. Cannot be automated via workflows in any apps, since the OS handles the Document Provider panel and the source/destination work.

So unfortunately, Document Providers won’t solve anything, and will just make the situation much worse. It has all the drawbacks of the share sheet, plus more complicated file management. :\

>“I wouldn’t want my databases syncing with two places at the same time”

This feature suggestion is not about syncing two places. It’s about copying/snatching (moving) files from watched cloud folders and putting them in your database. Each time it sees a new file it hasn’t seen in that folder before, it performs the chosen action (copy or move the detected file into your actual DEVONthink database) and that’s it. All cloud storage APIs have functions for watching folders and copying/moving files, so most of the work is already done.

This new feature would allow quick, automatic sharing between apps, with preserved filenames, and workflow support in scanner apps. All without having to talk to any 3rd party developers.

Unfortunately there are only 2 ways to automate integration between apps on iOS:

  1. Talk to the developer.
  2. Go via the cloud. The method I am proposing.

All others (share sheet, document provider) are manual processes and cannot be automated by apps.

>“why not tell us what app on iOS is your benchmark, wow-that-cool, design for an iOS share sheet”

There is no gold standard share sheet. They’re all slow and convoluted. The list at the top of this post lists some reasons: Share sheets cannot be automated by the apps, can only handle one file at a time, don’t preserve filenames, and necessarily need to manually select a destination within apps like DEVONthink To Go 2 since you need to choose which database to put something in. That’s just awful.

And Document Providers are worse; their purpose is to let you browse folders and edit mutually shared files in-place without having to go back to the other app and hitting “share”, but the downside is yet again that they can’t be automated, can only do 1 file at a time, and that the other app must have an intuitive file-folder structure for the other apps to browse, which DEVONthink definitely doesn’t have and won’t have (since it’s a database and needs to have its own internal folder structure that makes sense for it). DEVONthink’s folders are not meant to be browsed by humans. That’s what the DEVONthink GUI is for.

So all in all, I think having a watched folder via the cloud will give the most bang for the buck for people who only use their iPhones/iPads for DEVONthink, without having constant access to a computer. Many computer users already enjoy the “watch a folder” feature on the desktop. If it’s brought to mobile, the circle would be complete for mobile users too. This is a good thing. :slight_smile:

4

Note: There is now (as of 2.0.4) a new “open in” function “Import to DT” which is faster, see below.

Clever presentation/marketing technique. A cumbersome 9-stage process. Yet the first two are not really part of what we are counting here, neither are the last 4!

  1. No one can relieve you of actually scanning

  2. and generally, you also need to indicated that you want to share! People who scan 100% to an app are the minority. I distribute my scans to mail, DT and others.

  3. The “over and over again” is also not reasonable, because you also have to scan over and over again, and with the process I suggest below, the actual transfer is only a fraction of the time and effort you have to put in per document.

  4. Cleaning up: Yes, but so what. I do no trust any kind of automatic transfer to the extent that I have the document auto-deleted in the original app. I usually let them sit for a couple of days, and then do a quick bulk-delete of things that are not brand-new (kind of the function of the trash can in OS X or DT, but no such thing on iOS of course).

  5. Now the cumbersome step of selecting a database: seems fair enough, but if you hard code that into a workflow, you’ll have to do some sorting later on in DT; so little gain here.

Now: this discussion might be superfluous anyway, because in DTTG 2.0.4. I noticed that there now TWO “open in” functions in Scanner: “Clip to DT” and “Import to DT”. BOTH of the carry over the name of the document (so that’s fixed), and “Import” does not ask for a DB, but seems to send the document straight to the Global Inbox. That’s my preferred scheme.

So now we have the following flow:

-1: Scan
0: Share
1: Open In
2: Import to DT

So it is effectively 2 steps. I agree it should be one, if “Import to DT” could show up in the top level of “Share”. But given how much time and effort goes into the scanning (at least for me) - boundary adjustment, color, naming … - this is entirely reasonable and does not warrant any further, direct “deep integration” which would continually break with updates on both ends, and also exclude a host of other scanning apps that others might prefer.

Bulk import: Would be handy if available, but it you operate at that level, I suggest saving with a workflow in Scanner to Dropbox, and then import next time you have the Mac open or index the Dropbox folder. The way iOS is set up, it is not meant to do that kind of heavy lifting.

5

Just a quick note on this comment: This would be an untenable situation for any developer. Not only do cloud providers come and go frequently, they also change their APIs unexpectedly. Having to code for a watchdog APIs for multiple services would not be a good use of our resources.
Just something to think about.

6

@gg378: The steps I posted are accurate. Without workflows you have to click around to each destination, one file at a time, and repeat the Share process for every single file. And you say that you also personally repeat this “Share” process to manually send each scan to “mail and other apps too”. Very tedious.

With workflows, that’s all one step: Scan via the workflow and you’re done. The scan is then automatically sent everywhere you’ve specified in your Scanner Pro workflow - such as automatic naming, emailing, sending a copy to dropbox, sending a copy to devonthink (if DT is ever integrated via either dropbox or via direct local interoperability), and then deleting the scan from scanner pro itself. All happening simultaneously. Massive timesaver. And it works for multiple documents at a time. So you can make 5 different scans (without sending them anywhere), then select all of the final PDFs and send them all to the workflow’s destinations instantly.

Please, from now on let’s not further derail this cloud integration thread by trying to dissect the obvious merits of all-in-one-tap workflows. It’s been beaten to death in the scanner thread. It can be read there.

Thank you very much for pointing out that DttG 2.0.4 has improved the filename detection and has a new, faster import action (love it; I usually import to the global inbox and loathed having to scroll to save to Global Inbox constantly). I’ll try that right away. It’s a good improvement for sure. 8)

[size=80]Edit: Splitting this post since it was getting long by replying to multiple people.[/size]

7

@BLUEFROG: Yes, cloud providers come and go. Dropbox is the largest one and is used by so many apps and so many people that it’s not going anywhere. How about just supporting that one? They have a stable API for watching folders for changes. Most people use Dropbox since it’s the #1 cloud provider. Even my dear old mother. :slight_smile:

I’d prefer local interoperability (by apps speaking to each other directly) instead of having to go through the cloud at all, since I don’t want Dropbox to see unencrypted versions of sensitive scans. But Apple doesn’t allow that. The iOS Sharing/Document Provider methods all require manual intervention and navigation. The ways around it are either direct software-to-software integration (by talking to other developers and making your software communicate with each other in some proprietary way), or via bypassing iOS’s limitations by using the cloud. The cloud would circumvent Apple’s walled garden and allow one-step automation with endless possibilities, without having to manually integrate with any other developers.

So this suggestion about cloud folder watching was about much more than just scanning. If DttG2 could watch cloud folders and auto-import from them, it would open up all kinds of automated DEVONthink To Go workflows for all apps and operating systems that support the cloud.

Windows users with an iPhone/iPad (who have no ability to run a desktop version of DEVONthink) are plentiful, and they could make DEVONthink To Go 2 for iOS watch a folder like “/Dropbox/Send to DT”, and then just dump files in there from their computer, to instantly get them into their document repository. Cloud folder watching would open all kinds of wonderful possibilities and freedom. Cool huh?

8

OK, this is turning into forum ping-pong. I’ll bail out after this one.

Well, because real life is TEDIOUS? Because my work does not involve some scriptable bulk scanning (well, it does sometimes, but then I use the right tools at that moment, generally on OS X). For me, each scan is something highly individual. One scan is mailed to a colleague and has nothing to do with DTTG. The next one vice-versa. If that’s tedious, so be it.

9

Don’t ping pong then? If you’ve made up your mind that everything in life must be tedious, there’s not much anyone can do to change that opinion. However, the scenario you are just describing is exactly why you can create multiple personalized workflows:

  • “Archive Company Receipt”: Email to assistant, save to company dropbox, save to personal devonthink, delete from scanner pro.
  • “Share with Colleague”: Email to colleague, delete from scanner pro.
  • “Wife”: Email to wife, empty bank account, save to devonthink’s “weep” folder, delete from scanner pro.

:smiley:

Alternatively… you can ignore workflows and tediously do all of the above manually for each individual file - going through and sharing to each individual application, cloud storage, and email recipient, step by step, one scanned file after the other, until your thumb breaks from all of the tapping. Sounds like the plot of a black and white, silent french movie about the bleakness of life. All that’s missing from the picture is booze and cigarettes. Good thing we have choice, right? :laughing:

Most importantly of all: This thread isn’t just about scanning. This is about everyone who wants to automate data importing into DEVONthink To Go 2, from any source (app or OS) that supports the cloud. Particularly helpful to all of the unfortunate Windows users who have no DEVONthink for Windows.

10

OK, broke my vow :laughing:

Yes, we can have all our opinions. But consider this: Discussions like this happen rather frequently on this forum. In some cases, the discussion amasses quickly remarks like

“^^^ THIS”, “+1”, “+10”

and so on. You can completely ignore my patronizing. But judge the demand by how many people chime in for support.

Over and REALLY out. :smiling_imp:

11

You think that, really? Wish it could be counted on. Dropbox has changed APIs available to developers. From a developer’s standpoint, one hopes for prior notice of such changes in a good time frame, to allow for a maintenance update to resolve the changes. This isn’t as risk-free as you assume, for allocation of development resources. A related problem, of course, is that many users don’t take advantage of every maintenance release, so will be puzzled when an API breaks, resulting in Support tickets that may or may not contain adequate information to diagnose the issue.

Many DEVONthink users find that the data limits of free Dropbox accounts are not adequate, so would have to consider a paid account. In that arena, Dropbox does have lots of competition. We frequently see communications in the forum and in Support from users who are moving away from Dropbox for one reason or another, in choice for a larger account. Note that Apple is getting more aggressive with iCloud in Sierra (although we plead with users not to allow their databases to be placed in Documents or Desktop locations synced via iCloud in Sierra, or directly within a Dropbox or other cloud location).

12

What? The thread was created just 4 hours ago, on a pretty inactive web forum for hardcore fans, and the thread consists of a long and detailed feature description aimed at the software developers, only publicly posted so that the rest of the forumgoers can debate the idea. I highly doubt that this thread will ever see a flood of casual +1’s, since I didn’t aim the detailed feature description at the impatient “+1 crowd”.

13

@Bill_DeVille: It’s unfortunate to hear that Dropbox has deprecated old APIs without prior notice. However, the solution is still pretty simple:

  • Do the API call (I think the one to watch for folder changes is called “/delta”).
  • If the HTTP connection succeeded but the API call returns an unexpected result (ie no answer, faulty answer, unknown answer, “no such API”, or some other random gibberish), show a dialog box to the user, saying that “The Dropbox communication protocol seems to have changed. Please check for a newer version of DEVONthink To Go.” (In other words, the exact same thing you already have to do anyway, since you already rely on the Dropbox API for the cloud database syncing.)

However, they’re unlikely to change the API anymore, since each change hurts their existing base of thousands of integrated applications. If they do something, it’ll be a for a good reason. The past API changes were probably necessary to provide support for new Dropbox features, but by now, in [insert current year], they have most certainly settled on an API format they like.

As for other competitors: You’re right that some are growing, but nothing is even close to as ubiquitous as Dropbox, and shaking its monopoly is going to be hard. If something else ever gets big enough to challenge Dropbox’s near-monopoly on cross-platform syncing and ubiquitous application integration, then support watching a folder there too.

Windows users will thank you, since they have no DEVONthink desktop application, and would love a way to easily drop desktop files into a dropbox folder and have them show up in their DEVONthink To Go on their iPad/iPhone with minimal effort.

14

Noooo, I have to do it again. Well, I’m supposed to clean up downstairs, so maybe I’m looking for an opportunity to do “important work”.

I’m not saying that you should judge this by how much support has rolled in NOW. I’m saying: keep an eye on it. On top of that, your previous post voices the same sentiment and has been around for longer.

Btw: The “watching” of a Dropbox folder is probably something I’d use. We’re not totally at odds here. I’m more coming from the side of “what’s reasonable for the devs in a world of limited time and resources”.

Over and most definitely out. :blush:

15

I tried this out (sort of; to the extent I can) by saving files from ZoomNotes (and a few other apps that I use frequently on my iPad) to Dropbox. In an app like ZoomNotes I found it takes me just as long/as many steps to save to Dropbox as it does to save directly to the destination location in DTTG. So having DTTG sync with Dropbox in that case probably wouldn’t save time.

Of course, with other apps it would be faster – in Editorial the document I’m working on is located in Dropbox anyway, so it’s always being saved there. So the sync-with-Dropbox proposal would maybe work out.

I have a lot of folders in Dropbox and having to save to a “specific Dropbox folder” so that the documents would get from there into DTTG would probably break my hierarchical filing methods – or I would have to save documents to two places: the folder for my project and the folder for DTTG. I’d have to think about how that would work out.

I’m with Bill in being skeptical about Dropbox. I don’t know why they would or wouldn’t ever change the API again. They certainly haven’t always been trustworthy or transparent about their plans.

Hmmm – I wonder if there’s a WebDAV flavor that’s possible for what’s proposed by the OP?

16

Once again I need to chime in here. As I said previously, I’m all for automation where possible and can’t entirely disagree with what you’re asking for. Having said that, you’re asking for something that not even Readdle have done with their own apps. Sure, they have a button to quickly send a scan to Documents or PDF Expert…BUT can you automate a couple more actions in a workflow. Let’s take what you’re asking DTTG to do and try do that with Documents or PDF Expert…

  1. Scan a document (or multiple documents) then using a WORKFLOW, send and email, send to Documents/PDF Expert, delete original scan… Doesn’t seem possible. At least I haven’t seen a way to do that (and I don’t think I’m missing something). Heck, even the tight and direct integration between Readdle apps is disabled when you select multiple documents.

While I agree it’s a nice idea, you’re asking DTTG’s dev to do something not even the Readdle team has done with their own apps! I’d much rather they focus on improving DTTG with sensible features that serve the mass of their users than cater to a singular feature request that hasn’t even been attempted by the TEAM at Readdle.

That’s all I’ll be saying on this matter. Great idea I’d love to see in a plethora of apps, but one to be kept in the maybe/someday list.

17

@Steve J.

Thank you very much for being bold and daring with your forward-looking suggestions for improving the platform (iOS) and the DT Apps in particular!

It’s interesting reading the feedback and responses thus far when I consider how far we’ve come in terms of being able to compute on mobile devices. Where would we be if Steve Jobs and his team at Apple weren’t so forward-looking and adventurous?

So what if Readdle and all the others haven’t done it? So what if Devon Technologies are a tiny shop? Is that how we should determine what, where and how we focus our attention? Devon Technologies are in a very critical and unique place at this time and how they respond now will determine whether they go beyond being a niche shop or become a major player in this market segment.

I for one want them to thrive so that all my time with DEVONthink will not have been in vain. With rapidly advancing AI/AR Technologies, nothing is guaranteed. They tools we use and take for granted today will most likely be but distant memories tomorrow.

18

There is no basis for this claim. I’ve now followed computers with intense interest since the Sinclair ZX80 around 1980. I have not thought too carefully about it right now, but off the bat I can only come up with one thing in all those years that really came onto the scene in a somewhat unexpected way: the internet itself. Even there, the underlying network techniques were entirely incremental, and certain professions had used networks all along. But the way it was taken up by society at large was special. In that sense, Geocities was bigger than Facebook in my view.

Everything else has been a never ending stream of incremental improvements. Yes, even the smart phone.

Case in point: the scanners that are the starting point for this thread. I saw them in the 80s, my brother had one in the very early nineties. Have become better ever since, but that’s all. Now we can take ScannerPro on our phone to do it (and the quality is typically worse than a 90’s flatbed scan), but that’s not a revolution.

When I started university in the mid-80s, AI and “expert systems” were all the rage, and now we’re back to it. Let’s talk about it in 10 years again. At this point, AI is not even “AI” yet, in the true sense of the word.

I’d rather say that: Most tools we see coming, and finally they become powerful or small or battery-conserving enough that they enter our worksflows, but the older tools invariably keep hanging around as well. Nothing seems to have faded to “distant memory”.

Presumably, Ray Kurzweil would disagree, but that’s OK with me.

19

I agree with that, in a sense, there’s nothing new under the sun. I wrote my diploma thesis about VR which was the hype at the time. Then it got buried for almost 20 years and now it’s the hype again. Same with the various ways for text chat. I used so many in all these years but it’s still text chat, even if it’s now called iMessage, Facebook Messenger, or WhatsApp.

But technologies come and go. And new developments can disrupt existing ecosystems. The rise of mobile phones with advanced cameras destroyed the compact camera market — without giving the camera makers a real chance.

20

Absolutely correct, Eric. Just as you point out with the texting apps, the particular breed of point and shoot camera might be distant memory at some point, but the activity of taking pictures, even on a phone, is still remarkably unchanged, even from pre-CCD days. So even if devices leap, the “experience” is evolving, in most cases, rather smoothly.