Unreadable attachments on iPad and other bad surprises

DEVONthink
, ,
1

I recently discoved that some very important PDF documents (attachments) were UNREADABLE ON MY IPAD. It was only some notes and I could find no clear pattern. - So I started testing with brand new notes on my Mac and synchronized to iPad to see if this was OK. And I got a lot of very bad surprises. I could repeat the problem in lots of ways with current Devonthink software on current Apple hardware/MacOS and iPad Pro 11”/iPadOS with brand new files created for this purpose. During this test I discovered that there ARE ALSO PROBLEMS ON MAC.

I use both Plain Text notes, Rich Text notes and Formattet text notes. And I attach files to those notes. I use PDF, JPEG, PNG, Numbers and Pages files as attachments. Sometimes I create a “new note…” and then drag the attachment into the note body. Sometimes I simply drag a file from Mac desktop directly into Global Inbox. And sometimes I use the Apple Share menu to bring a file into Global Inbox on the Mac. I may also use Share on iPad/iPhone with DTTG.

And then I synchronize to/from my iPad/iPhone and our two other Macs. And expect all to be well and that I can bring my iPad along and have my documents with me. - But that turns out not to be true. At least not on my devices

The tests I made was to create notes from scratch on my 24” M1 iMac in all the possible combinations I use. And then synchronize to an identical iMac (wife has a green one) and to a MacBook Intel 2019 and to my iPad Pro 11”. All devices use current MacOS/iPadOS and current Devonthink 3 version and current Devonthink To Go version. I created all the test files myself, so there would not be any “funny stuff” in any of them.

Synchronization between the Macs went well. They are all identical. Synchronization to iPad also went well in the sense that all notes were synchronised and showed up in Global Inbox on the iPad as expected. - But when I started examination of all the notes on both Mac and iPad things turned out very badly. - You can see the total test results for each combination in the figure below. Red and yellow are (in my opinion) bugs. White is “OK, as expected”. Green “I am surprised, but happy that this works”.

About a week ago I wrote about this to Devonthink Support. But it is holiday time and there may be a long list of issues before mine. So I have not heard anything from Support yet. Things take time. I was a supporter myself, so I understand that. But I decided that perhaps this was of interest to the user forum. Either other users can repeat my issues or I may come to know that I am “all alone in the world” with this. Because I need to do “something” no matter what the situation is. I cannot continue trusting my data after having seen this.

Perhaps somebody can repeat the test or find some consoling words for me. Using attachments in Devonthink turns out to be both complicated and a risky business too. I wonder how to find the “bad notes” which are already hidden somewhere in my 3000 notes of which more than half have attachments.

Devonthink test results
Devonthink test results1920×2046 358 KB

2

I suspect it could be helpful to mention what method of synchronisation you are using. At first glance, I didn’t notice any mention of that in your post.

Stephen

3

I live in an all Apple home, so it is iCloud (cloudkit). But the problem does not appear to be related to sync method, as all data actually are sync’ed. Both to the two other Macs and to the iPad. It would be a very strange sync issue which could produce all the different test results.

There are many older notes which do not have this problem. It was to eliminate all data and sync problems that I created everything from scratch. - So I am looking for somebody who can confirm that these issues are reproducable on their devices also with current software. - But, of course, who can tell if this is only related to the way Devonthing synchronizes, the method used may be of relevance. But I can only test on iCloud (cloudkit) sync.

4

If I understand your description correctly, your expectations are a bit off the mark.

Text files

As their name implies, they are just that. Text. Characters, meant to be read by humans as they are (the characters and the humans). When you “drag” a PDF into a text file, what exactly are you hoping to achieve, given that PDF is a binary format not being meant to be read by humans? If your PDF only contained text (which it most probably does not), you could perhaps hope for this text to be added to your not as it is.

The same goes for any binary format: Images in nearly any format, Numbers, Pages, audio… There simply is no sensible way to add them to a text file. So, some software tries to add a link when you drag the file into the text document. Personally, I think that’s a stupid idea, but one that might seem to be appealing in the AppleVerse.

Stupid, because the link is to a local file, i.e. one on your disk. If you send the document to someone else or simply share it to another machine, the link will be broken.

Formatted note

A formatted note is an HTML document in which you can embed images (well, you’ve found that out already. It is also explicitly mentioned in the documentation, though). So, HTML. Which is another text format, and which cannot embed PDF. Simply, because there exists no HTML element that can display PDF (as opposed to images and possibly video/audio with modern browsers!). The same, again, holds true for files from Numbers and Pages.

Therefore, again, the software might try to add a link to the documents it can’t reasonably embed. Which, again, must point to a file on your local disk and makes no sense whatsoever on any other machine.

Rich text format

That is the only format actually providing attachments. Which you have already figured out.

What’s wrong with the links?

A link consists of a protocol, a host and a location part. The protocol can be something like “http”, “file” or “x-devonthink”. The host (aka server) can be empty for some protocols, like “file” or “x-devonthink”, in which case the location refers to something that either the browser or the application understanding the protocol (like DEVONthink) can find locally.

Here’s what I get when I drag two objects from my desktop into a text note in DT:

/Users/ck/Desktop/Bildschirmfoto 2022-07-15 um 10.04.21.png
/Users/ck/Desktop/SEPA-Lastschriftmandat einmalig Kopie.pdf

As you can clearly see, these are not links, since the protocol is missing. They are simply locations in your local file system. Nothing to click, consequently. And, BTW, the exact same thing happens with a Numbers document.

And, if you were getting real links, they’d look something like ‘file:///Users/ck…’, telling the browser to open them. Which would be utterly pointless in the case of PDF and Numbers.

I suspect that the drag & drop as it stands now is independent of DT: If you drag a PDF to a document in Text Editor, you’ll see the same behavior. In implementations of Drag & Drop that I’ve seen, a drop site indicates which kind of data it can receive. In the case of a text document, that can’t be anything else as text. Consequentially, the service converts its data to text – and the closest to text it can provide is the file name.

Conclusion

You’re not confronted with a problem in DT, but with mismanaged expectations. If you need PDFs, Numbers, whatever documents in DT, import them. Then they’ll be synchronized with your database(s) on the iPad and will be available there.

6 Likes
5

You could, of course, use Bonjour. Which is faster and more reliable than iCloud. But your problem is not sync related at all.

1 Like
6

I have not tested each of your cases, but those which I have tested, on the whole I can confirm your results.

Re. New formatted note (PDF, Pages): same result, although on the Mac I can click on the file:/// link to open the file. The same does not work on the iPad (obviously, the link is to an external file on the Mac).

Re. New Richt text, Mac (PDF): embedded as, and opens as, a PDF+Text. I can’t figure out how I would open the attachment in anything other than Preview, though.
Re. New Rich text, iPad (PDF): same result (unexpected)

Re. Number and Pages files on iPad: Pages does not save the edit back to DTTG, but rather drops it in iCloud Drive. I don’t think that is in DEVONtech’s hands; not all software is compatible with edit in place.

All in all, the results are close to what I would expect (see @chrillek’s explanation); I would probably expect a more detailed context menu for an embedded PDF in a Rich text, and I would expect the embedded PDF to sync to DTTG. I find myself wondering whether DT should prohibit dropping non-supported file types (e.g. PDF into formatted note) to avoid confusion.

3 Likes
7

Thank you very much for your detailed answer. It is a great help.

What is a “bug” to some is a “feature” to others.

I think you explain things well as seen from the app’s point of view. As some of the pdf options are “green” it will be possible to use pdf’s even as I expect them to be used. At least it will be possible for advanced users who know that this is what is possible and knows what should not be done.

But I did this test from the view point of a casual user. Because that is what matters in our home. My wife (66 years old) is such a casual user. Absolutely not an IT person. But she takes a lot of notes of various kinds.

Years ago we began using Evernote on our Macs, iPhones and iPads. That was easy enough. You created a note, draged a file into the note and afterwards you could maintain that file (like our budget Numbers file) from within that note. It was simple and worked well.

But then Evernote decided to rebuild itself from the ground up. It took years. So while this was going on we moved all our data to Keep It (Keep It Mobile on iPad/iPhone. This also worked pretty well and the level of (actual) bugs and support was OK. Recently I found that the present versions of Devonthink and DDTG was stable and had some nice options on top of what is found in Keep It. So we moved our data again. My own IT background makes this possible without too much hassle.

The wife had been reasonably happy using Evernote. And more happy using Keep It as it is very “Apple like”. Much like Apple Notes to her but with more options. Also here it was possible for her to just create a note and add an attachment. No extra wisdom required. No need to “think like an app” which would not go well. Trust me. For her a “note” is a “note” no matter if it is “plain text”, “formatted” or “rich text” or one of the other types. So if she can add an attachment to a “note” she will do just that.

With Devontink the wife (and I) continued to create notes and add attachments as we have been used to. We have not really paid attention to the app’s internal logic concerning valid and non valid data types etc. If the app allows something it is OK to do it. If it should not be done according to app logic then it should be the app’s responsibility to make it not possible. Preferably with a nice little explanation and suggestion. At least according to our expectations. Like an Apple app would do.

So even if every test case can be considered as either a feature or something which a smart user would know not to do, these colored test cases will be considered as “bugs” from my wife’s point of view. And that is the point of view which matters in my home!

Your explanation tells me that the wife and I are using an app which is not for us. We can’t all the time be aware if we are using precisely the same method which the app wants us to know and remember. We are not nerdy enough, just “ordinary end-users” who want to store all our data in some kind of database-like app where we always can find them. Like we were used to with Evernote and Keep It.

As the test cases apparently are not seen as “bugs” from the Devonthink app’s point of view there really is only one solution for me. Move back to Keep It so the wife (and I) can keep doing what we have been used to do without having to worry that something gets “lost” on the IPad because we chose an “illegal” method.

Lucky for me I still have all the Keep It data and a subscription for all devices. So with hard (and careful) work at the keyboard I will be able to pick out all the notes created directly in Devonthink and move them to Keep It. And I will probably also be able to “fix” the missing attachments. Because I never really deletes things. I just park them in some remote place in case they are needed. Like they are now.

So some time in the near future I will have 3 Devonthink “seats” for sale…

8

Thank you for your reply and testing. So we know that the test results are not a local issue in my home. But I guess that you are right about about expectations. I have carefully read chrillek’s answer and responded. My conclusion is, that Devonthink simply is not the app for me and my wife. While the app’s behaviour may be reasonable enough when seen technically from a computer’s point of view it is not reasonable when seen from a wife’s point of view. And that is what counts where I live! So we move back to Keep It. - See more in my response to chrillek, if you like.

9

We have 11 Apple devices between us (Macs, iPhones, iPads, Apple TV box, Apple Watch) using the same Apple Id and iCloud synchronization. And because we live in a country where we at least have a 4G mobile connection everywhere the synchronization also takes place when we use our iPhones away from home. We have lots of apps which synchronize this way so changing sync method is not really an option for us. Besides, I would never be able to explain to my wife what “Bonjour” is apart from the obvious french meaning.

10

I have read your response to @chrillek. There’s nothing wrong with deciding that the tool at hand is the wrong one for the job - and in consequence picking a more suitable tool. If you feel so inclined, do post more details on your use case though. I’ve never dragged a document into a note before today (and never got on with Evernote, either) so it’s unlikely that I currently have an understanding of what you are doing. So, as I said, and probably more for my benefit that yours, feel free to elaborate. If I recognise things you are doing, I’ll come back and post how I do it.

1 Like
11

“Using Bonjour to keep the devices in sync means that the process runs across our network at home, rather than across Apple’s network which is iCloud. It’s like the difference between shaking hands and using video chat.”

Anyway, there is no reason to use another method of sync if things (sync related) are working. I think it was more of a hypothetical possibility.

1 Like
12

Evernote “notes” are containers, not files. DEVONthink has something similar (though not identical) with RTFD (i.e. rich text with attachments). So, it’s no wonder that your expectations are not met.

3 Likes
13

Indeed, there is a lot of smoke and mirrors in Evernote.

14

Just to follow up.

When I left Keep It it was on a specific date. And I moved to Devonthink the day after. So metadata dates could be helpful later on. Happy for that now. I also added tags before moving from Keep It. - The move was made as export from Keep It to HTML and then import to Devonthink. I kept the Folder/Group system too. - So now I have been able to move all “new” Devonthink back to Keep It as it was the day I left it. I could actually take all “new” notes in Devonthink and just drag them over to Keep It, no matter what note type and what attachment type. It all worked perfectly. No rejections from Keep It. - Big smile here.

While having Keep It open I did exactly the same tests with Keep IT as I described for Devonthink. Same data, same methods. - And all was synchronized to Keep It on the same iPad as Devonthink. - All 15 cases were a success, both on the Mac and on the iPad. The attachments were editable on the iPad and the changes were synchronized back to the Mac version without any problems.

In Keep It you can make various note types like you can in Devonthink. And you can add the attachments to any note type. Because Keep It handles the Attachment as the file type it actually is and the note itself as the type this particular note is. Pretty much like it was in Evernote.

And that is why we, as two end-users have been accustomed to “anything goes” and that the app(s) warns us or rejects things which won’t work all the way between the Mac and the Mobile versions.

Of course I now know that my/our expectations from using Evernote and Keep It was not met in Devonthink. Nothing for me to do about that other than move back to Keep It, which I have now done. - But it will continue to be my personal opinion that it should not be the job of the end-user to remember all sorts of technical issues and limitations when using apps made for end-users. - The developer(s) should do all the hard work once so the end-user do not have to do it all the time. I have worked in software development companies where we followed that rule.

But, of course, everybody is entitled to their own view on these matters. Some people are probably happy that Devonthink works this way. We are just not in that group.

15

Last time I looked, Keep It used the same container approach as Evernote, Bear and Obsidian. So, everything looks honky dory on the outside, and more power to you.

Let’s just hope that you’ll not be frustrated if you ever have to or want to move away from that app. If and how you the get at your data in the next one is anybody’s guess. DT (mostly) stores your data as they are, without containerizing them (with the exception of formatted notes and RTFD). But again, it might just not be the right tool for you.

You’re of course entitled to your opinion. But frankly, expecting an app to do what you think is right with any file format you throw at it, is a bit too naïve. Try drag-dropping a pdf or a Numbers document in text editor. What do you see?

If you can’t be bothered to understand the different formats and their limitations (and that’s your right, of course), then stay with the one that does what you want.
But don’t take it out on an app that behaves perfectly reasonable within the technical constraints of a given file format.

2 Likes
16

That, presumably, depends on the end-user the product is aimed at. If I, for example, purchase a book covering an area within my specific professional field, I expect it to have content which would be incomprehensible to a layperson.

3 Likes
17

Indeed not every app fits everyone’s tastes or ways of thinking. Even though DEVONthink can be used in very simple ways, the simplicity may not be obvious at a casual glance. And it goes both ways… for example, apps like Obsidian or Notion or Craft get no traction in my life outside occasional support questions.

1 Like
18

Well, we now know that I am a “container type” person and not a “file type” person. So thank you for confirming that I am on the right path.

I actually don’t expect an app always to do what I think is right. Although it would be nice. But I do expect the app to make sure to reject something is is not capable of handling. In particular when one of the products is a “to go” iPad version where user expectations are that “it just works”. And that is why I am a bit disappointed having bought three seats to Devonthink. Because it turned out that my data really was not there when I was “on the go”.

Of course it makes no sense to drag af Numbers document or a pdf file into a text editor. But it is possible to treat the note part of the data and the attachment part of the data as the different data types they are. Lucky for me this is what happens in Keep It and Evernote. Lesson learned.

My views on how an app should be designed to communicate with a “normal person” goes back about 40 years. And in jobs where I had a say on things I have practiced that view. - But obviously I won’t bother Devonthink fans with this any more. Having abandonned Devonthink it will also be time to leave this forum.

So Roger, Over And Out from me.

19

Feel free to come back in the future should you choose to explore things further.
You and your wife have a great week!

20

I share that expectation.

Again, I think the design can be tailored to the audience. There is a fine line, of course. If I imagine a ventilator on an intensive care unit, I would assume it does not have to be designed to be operated by a layperson; at the same time, alerts (think “patient disconnected!”) should be obvious and clear (so “patient disconnected” vs “patient not ventilated” vs “error 13”). I’m not sure who you are, nor who DT’s audience is, so this is just hypothetical rambling.

All the best! Please permit me one final piece of advice: one of the advantages of DT, and its foregoing a container design and proprietary file format is that you can always get your data back out again. I can access all my files even if DT stops working today. Make sure that for whatever software you end up with, that is and remains the case.

1 Like