DTTG 3 is corrupting files

Are you serious? You have to pull this app from the store!

Absolutely!!!

I don’t presume to have enough information to make that claim. I actually do trust DT and will reserve judgement for a later point. I have an appropriate backup strategy and always assume my data will be lost. I accept that to my knowledge I am not affected by this bug, and might feel differently if I were.

Based on the information available at the moment, I have set up a smart group with conditions to determine whether I have been affected by the bug:

All of

  • Kind is Any Document
  • Size is 0
1 Like

do you mind sharing your (entire?) backup strategy, pretty please? without time machine, how do you get versioned backups? trying to set one up for a few weeks now but especially the cloud backups turn out to be quite tricky (for me)

edit: discussion here: Backup Strategies

1 Like

@eboehnisch @Blanc

I’ve made a quick pairwise comparison on file level between a database containing records with ‘0-bytes’ and a previous backup, although those records were in the global inbox and moved towards another databse in the meantime.

This is certainly preliminary, but it appears the ‘0-bytes’ files are duplicates of the original and if that is correct it would be fairly good news as the originals with content (>0 bytes) seem to be available in the package.

For those who are unfamiliar with examing the database file in finder, please be aware you might damage the contents if performed incorrectly. So if you do check, you obviously do so on your own risk.

1 Like

@Solar-Glare is right, and I’ve moved my response to a separate thread. Let’s discuss backups there :slight_smile:

1 Like

If I may barge in: I used a slightly altered version of the script that comes with DEVONthink to save the current database as a zip file. I attached it to a remainder to automatically do so every week and I launch it manually before every major re-organization or update.

The zip files are saved both in a cloud and on an external HD. These are just snapshots and their usefulness depends of the intervals between the backups of course.

you most certainly may :slight_smile: @bosie is looking for ideas, and I’m not selling anything. The important point is that every user should have a backup strategy suited to the importance of their documents and the time they can invest in restoring. Thanks for sharing :+1: Let’s continue the discussion here.

1 Like

If people don’t mind, I would suggest discussing backup strategies in another thread and leave this one for the topic on hand: do the files actually become corrupt?

Or are they (based on my quick and prelimary investigation) somehow duplicated and does one of the duplicates turn out to be 0-bytes in size, which are erronously presented as the primary record in DT?

2 Likes

Some users have been discussing the same problem for a couple of days in another thread (Incomplete sync in DTTG3?). What you are suggesting is what I am hoping for, too. Maybe the corrupted files could be made accessible again.

I actually have had no problem at all. The new CloudKit syncing for me is rock solid. Using that smart group search didnt pick up any 0 size files or corrupted files.

I had been thinking Devonthink had been doing an amazingly good job and am really pleased with the reliable and faster sync.

Just a thought…but maybe you haven’t set the sync up correctly somehow ? It seems to work perfectly for me.

2 Likes

Who knows, but if you’ve only seen white swans, that doesn’t mean other people could have seen black swans. I.e. you cannot deduct whether there is a bug present that causes this behavior because you didn’t experience it.

@Blanc: do I understand correctly you experienced the existence of 0-bytes files? Are you comfortable with closing DT completely, creating a copy (that’s important) of that erronous database and inspecting the content of the package? It’s not terribly difficult to understand the file structure, but as said be careful to not make any changes might you want to use that database file in the future as you could corrupt it if you alter it by accident. Hence the importance of creating a separate copy first.

If you’ve opened the package in Finder, browse to the content folder (it’s full with subfolders describing the various file types by extension like pdf or jpg) and search for the filename of the 0-byte file. If my findings are correct, you will find the 0-byte file in duplicate: one with a file size of 0 bytes and the original with a normal file size. As said I don’t know what the impact of opening the file could be, so please take into account any risks involved.

No, I’m sorry if I was unclear in what I wrote; I have not experienced this problem - to the best of my knowledge my databases are intact.

(your instructions are immaculate - thank you for playing safe; I’ve previously looked at the file structure of DT databases. It was one of the things I did before committing to the software - I needed to be sure I could extract my data if all else failed).

As someone experiencing this bug, I can say that, at least in my case, it only affects DTTG3. Neither the main database on the Mac nor the data in DTTG2 (on the same iPad) have any issues at all. At least in my case, it appears to be a problem between DTTG3 and the sync store, not a problem with the sync store itself, and therefore not a problem that will propagate to other copies of the data.

Not to sound insensitive, but it seems to me that people who don’t have backups for large amounts of critical data have a separate problem that DevonTechnologies can’t fix.

Katherine

2 Likes

Oops! Hopefully a fix is soon found.

We’re already working on a hot fix and will submit it to Apple tonight.

10 Likes

Will this fix only prevent “new upgraders” to get into trouble? Or will it also “cure” the actual corrupted files?

1 Like

That sounds like you gave it a high priority! :+1:

Any details on what went wrong and am I correct the files aren’t corrupted at all, but somehow get duplicated with a 0-byte size?

2 Likes

I’d (obviously) be interested in what activity causes this.

I only use DTTG v3 and created a DB from scratch that syncs between iPhone/iPad via CloudKit and Bonjour.

So far all of my (several hundred) PDFs are fine as fine as far as I can tell.

1 Like

We believe that the problem occurs in a method that is applied to PDFs when you open them. It was actually suggested by the makers of the PDF engine and corrects highlighting and underlines that PDFKit on the Mac adds in a way that goes against the Adobe standard. When saving the corrected PDFs, for a yet unknown reason, writing the file fails and creates an empty or corrupted file. At least, that’s the theory for now as we have no device here that shows that problem in the labs.

We have now temporarily deactivated this method until a) we have made some further tests and b) added fallback code that keeps a backup copy of the PDF and jumps in when saving the PDF fails and replaces the corrupted copy with the backup.

Expect a 3.0.2 as soon as it has passed Apple’s review and a 3.0.3 later this week that hopefully reinstates the corrections.

What we don’t know is why it didn’t hit a single beta testers in many months and why it hits anybody at all because the code wasn’t changed in a long time. So we believe it must be a side-effect of the migration from v2 to v3 — but all beta testers went through that too. And it only hits some customers, not everyone.

When you receive 3.0.2 please let us know if it fixes the problem for PDFs you hadn’t opened before in DEVONthink To Go 3. The fix does not resurrect already destroyed PDFs. They are gone and you’ll need to replace them from your backup.

Please accept my apologies for this very unforeseen problem.

9 Likes