DTTG 3 is corrupting files

I would, but I do not find it. Using WEBdav on a Synology with DSM 6.x. In my DEVONthink folder, I have
.DS_Store DT.dtCloud @eaDir #recycle
and in DT.dtCloud I see

09b8bafd2d664164cac0aeab533876724187c0f65e56255174ce5f25903b375c
4071363aacbeb8ea1ad26584a4ec9463c3fa46d3bbc31acf0ac0955569ba9883
7cba81b0c71fc3f8b7135c7d6a933cd51baecb26d8c3cf66c23197c51db572e5
7ce2ed434f942e4d75d0a25aa5a3cb663163042e75cf948dc3ada0ec4f1c7a29
9e99eeb4607550a68d0c330d7ec6685542067d0f4f91d78e06004ff6a193f669
c37f1dfa93bf26dd2c873af5546e6f163a3f01d02e7fdbec7738e5cfcd9eb98d
e2cd4d73d5b9ace5b7a4d75333e5d3a3e054201cff3f539ca3bc8491874b3784
inbox

All these entries are directories. I did a find . -name .dtSyncStore, which didn’t turn up anything.

Thanks for the 3.0.2 update this morning and the clear instructions. I was able to sort out (replace or delete) the ghost files before my coffee was done, turn automatic synchronization back on, and resume marveling at the powerful database in my pocket.

5 Likes

Thanks for all your efforts on this issue.

While I did have 13 ghost files (love that term, btw), most of them were some variation of “untitled.md” or equivalent — so my suspicion is that they just happen to have been flagged by your ghost criteria, and aren’t the result of anything to do with upgrading from DEVONthink To Go 2.x > 3.0.

Onward and upward!

Thank you for this excellent idea. We are just exploring this option. We added an automatic verify & repair on app launch which also restores ghost files from the V2 data store if possible.

3 Likes

Assuming that would be implemented, what would happen if a user deleted DTTG 3 from the device before the legacy data had been purged-would it be orphaned on the device indefinitely?

No. The shared container only lives as long as at least one of the two apps is installed. When the last app having a grip on the shared container is deleted, iOS cleans up.

1 Like

Thanks too @Magusomo, who went hunting and posted the necessary input - I just put 1&1 together.

This is a wonderful example of what a community working together can achieve. Everybody who has patiently examined their data, posted ideas and feedback has added a piece to a puzzle which isn’t yet complete, but which is starting to turn into a picture. I’ve said it before, but it bears repeating: this forum is a nice place to be. Thank you all for your input, helping to make the software we all use better. And thank you Eric & team - it’s obvious you’ve been at this tirelessly; thank you for openly sharing your ideas on what is going on, opening an opportunity for us all to look at specific questions and help where we can. I wish you luck & speed going forward - you’ll get the culprit in the end :slight_smile:

15 Likes

I have installed the DTTG update from today. I am using legacy icloud sync. The new DTTG update shows me 1 pdf in the “ghost-document” intelligent group. I see the thumbnail (0 file size). It was added on 8th January 2021 (I do not remember if I have added it on the mac or on the iphone then). I can open it on the mac (there I have no 0-file-size documents) but not on the iphone. I deleted it on the iphone. I cleaned the trash on iphone and mac. I was able to restore it from my mac. Just in case this information helps @eboehnisch. Is it safe to continue working with DTTG and DT for the moment? Or might it be that such 0-filesize documents reappear during sync process or database use?

As we are working with DEVONthink To Go 3 now since last April we deem it safe to use once the migration is completed and all ghosts of the past exorcized.

1 Like

I do not know if this matters… exactly this file has been searched for on my mac with spotlight today and was opened from spotlight there… I just try to understand why exactly this file has become a ghost file on my iphone (and none else) and why it is still existing on the mac.

(due to time pressure I am not able to read all the thread all the time… wouldn’t it be great if DT has an official thread where only the relevant information “what is known up to now” is posted for all users, pinned to the top of the forum… would help a lot…I am really afraid that I might miss anything or doing anything wrong… thanks so much for having a close look at this issue and for your transparent communication)

This is, indeed, interesting information. We still believe there might be multiple factors at play here, one being the copying of the data store from V2 to V3 (even though I cannot imagine any scenario where this should fail) and synchronizing with an old sync store that contains gremlin data from the distant past. There was a bug in an older version of DEVONthink To Go that theoretically could produce such an effect but we couldn’t make it happen here on any combinations of devices. So we’re now concentrating on the more practical approach as suggested by @Blanc: Repair what we can and keep problems from spreading.

We will soon put up an official pinned post.

Hi Gang,

I just downloaded DTTG 3 and it made the Ghost smart group. It found 1 document - it was an HTML page - Not even sure what that is from - its also on my Mac. Its not an important document and may be actually totally unrelated to this issue, but I figured I would let you know.

I deleted the HTML file.

The Ghost Smart Group was NOT recreated on the Mac, I just searched for the document. Is that the expected behavior ?

Thanks

Yes, the Ghosts smart group exists only on iOS and is switched on when ghost documents are found. Version 3.0.3 will also try to repair them. Did you migrate from version 2?

I also have an .mp4 affected…

In my Ghost Group is just one item, a webarchive, which can’t delete.

What happens when you long-press the ghost item and choose Move to Trash?

“move to trash” is colored in red, when I press it, nothing happens, the file is still shown.

My files took a minute to delete, just fyi. I let it sit a minute, exited the “Ghosts” smart group, then went back in and the files were gone. I kept getting the Ghost popup in DTTG3 for a minute (almost like it kept detecting the same ones or something) but then I went into the trashes area and cleared those out and that stuck.

@eboehnisch I successfully deleted my Ghost files, (that fixed my “failed database verification” error I was getting on my iPhone) and will manually restore later, but I didn’t notice that my files were 0 bytes until just a few minutes ago, and have been using DT3 on my Mac all morning, adding a few files here and there to my Global Inbox.

Even with all my databases verified and repaired successfully, those files I added this morning on my Mac to my Global Inbox are not syncing to my laptop or iPhone.

Should I choose “Rebuild Database”?