It’s simple DEVONthink functionality, frankly…
Thanks again. The thing is: when creating a smart group from the menu, it shows the option databases (plural) but after creating such a smart group, it reverts to database (singular). So indeed the only way I could create this database was pointing to the smart groups, and then use the contextual menu. Now I see all my corrupted files (I hope). Not many, thank God, but it does affect markdown and movie files too. They were created from October 2019 on, and most of them I haven’t opened (let alone moved) in many months.
Thanks for that - confirmed that behaviour here too; @cgrunenberg I think that is a (minor) bug in DT3.
Can you remember how they were created? And presumably you have synced your Mac with DTTG3?
Bringing into action my suggestion about ‘pseudo’ smart groups, one could of course also create a smart rule to replicate the 0-bytes to a group called ‘Warning’ or something in the Inbox or some other location you review periodically.
Replicates don’t cross database borders though, so it requires multiple rules to replicate to the various inboxes I presume.
Update: I understand DTTG 3.0.2 has such a feature build in. Great work. Didn’t see it as I’ve quit using DTTG momentarily.
Be aware that I’m still uncertain about the warning that was posted in the tweet (Feb 12th) that no PDF should be touched at all if a zero-byte file is found. One obviously does not want to propagate the problem by creating such smart rules. Is that warning about not opening any PDF still actual?
I agree that @BLUEFROG and @eboehnisch might consider writing some preliminary mitigation strategy and post that sticky to this board. Backups for example can have a limited time span for some and if people overwrite their backup files with the zero-byte files after some time the original could become unretrievable. If you use WORM media overwriting is impossible of course.
As mentioned before: these zero-byte files might have already existed in your database. But personally speaking I can find files that are of some size in a fairly recent backup and of zero-size now.
I’ve installed DTTG 3.0.2 and it has created the Ghost group, which thankfully only had one entry.
I’ve restored the file on the Mac from a Time Machine backup and resynced on the Mac end.
However neither the iOS or iPadOS version will let me sync. When I try to sync in DTTG I see an error in the log which says “Global Inbox: Failed database verification, please repair the database”.
Does anyone have any suggestions?
I know that at least the movie files were added from the Mac. The other files: could be both from iOS or Mac. And yes, I upgraded to DTTG 3, almost immediately after it became available.
@eboehnisch would have to comment, but that may be what is meant by this sentence in the change-log: “This update detects these files, shows them in a Ghosts smart group, and stops synchronizing the database.” I suspect the database will not sync until the 0-size file has been removed from the iOS device (that is trashed and the trash emptied). I’m not sure what will then happen to the file on the Mac - it may also be deleted on sync, such that you would then have to put it back from the backup once more.
1 Like
so there goes the theory that this could be affecting files added via DTTG2 only.
Ok thanks I’ll try that now. I’ve also submitted a support ticket so that you have the log files.
thanks goodness I won’t - I wouldn’t have a clue what to do with them, I’m just some random guy 
but Eric will have them, and that’s a good thing 
1 Like
My bad Blanc, newbie here, not sure who is DT and who is a random guy like me 
That DEVONtechnologies guy would be me, the random but insanely helpful guy is @Blanc 
Thank you for sending the logs.
4 Likes
The accounts of the DT staff carry a small green badge of a nautilus shell on their icons, that extends the actual icon.
3 Likes
What I don’t unterstand until now: I have 457 corrupted Files in two databases. One of the databases is the global Inbox which is created by Devonthink. The creation date of the corrupted files vary back to end of 2018.
This was the time I mainly switched from a Mac based to an iOS only Workflow. I created PDFs via a ScanSnap (and its OCR engine). These scanned ones and digital received pdfs (like email attachments) moved into Devonthink via the iOS files App.
So in my feelings, the corrupted files come from this iOS based workflow.
I have DTTG 2 still on my devices, alongside to DTTG 3 at the moment.
In my case I have the corrupted files spread in all DTTG 3 instances and also on my Mac DTPO 3, think because of the sync.
In the DTTG 2 instances I can access all these identified files! (The sync was disabled through the upgrade process to DTTG 3 I think.)
So maybe there is a possible way for you DT guys to make a recover routine?
Edit: I also just remembered that in two or three cases I had the case, that this iOS workflow via the files app didn’t gave the pdf file over to DTTG 2 correctly, because there weren’t accessible in DTTG 2. So I did the import workflow again over the files app. After the second try they were accessible in DTTG 2.
@MauriceB’s post further up suggests things are not quite so simple, or are multifaceted.
That is an important piece of information, especially considering that - if I understand correctly, that legacy data is still there even if DTTG2 has been deleted; DTTG3 deletes it sometime down the road (where on Earth did I read that? But I did…! Here; @eboehnisch I’m sure you will have thought of this long before I did, but is it possible to preserve that legacy data for the time being? I’m kind of assuming it might be possible to retrieve “lost” data from that source, even if DTTG2 has been uninstalled? Would DTTG2 “see” the data again, if it were reinstalled?)
Same here: The only file affected on my side was one added on the Mac, synced to DDTG and then opened via the share sheet in GoodReader on the i*OS device. Added a hightlight there which got transferred back to DTTG. And sometime after that the file size went downhill. Of course, having a backup helped.
Maybe you’re right @Blanc and I can’t be completely sure because it affects a lot of files over quite an amount of time in my case. But I think the important thing for me is, like also @chrillek said before, that it affects files which run trough the iOS share sheet. In my case my iOS workflow always went over this:
For the file usage on the iPhone Memory I also noticed this in my case a actual usage of 27,25 GB, but before the installation of DTTG 3 and the upgrade it was about 14,5 GB.
would you have had any reason to use the iOS share sheet on these files? That is sharing from or to DTTG?
I can concur that at least one of my 0-byte files still contains it’s data under DTTG2 (which I didn’t remove either) and that file hasn’t been moved from it’s location.
I must also say I was a bit surprised the syncing took quite long following the DTTG3 upgrade. It looked like DTTG3 started migrating all kind of files, but there didn’t seem any need to do so.