I just get shocked, when I found out, that multiple groups/documents where missing. All account statements, which I had grouped per year, are no longer existent. I suddenly found out, because I needed an account statement from last year, and there I saw the disaster.
I don’t know, if this has something to do, but I just opened my DevonthinkToGo App in the Tablet, and there I saw the message, that the App was “deleting 1205 documents”… I never changed 1205 documents in the last months, neither did I removed them in main Devonthink desktop application.
Can you give me a hint, how Devonthink removed documents on its own?
DEVONthink won’t delete documents without user intervention of some sort, whether that’s a smart rule, moving indexed items in the Finder, using external apps like Hazel to affect a database of indexed files, etc.
My guess is that the deleting in DTTG was just reflecting something that had happened before on DT. Simply syncing, so to speak.
So my question is if you had recently seen any inexplicable spinning beachball in DEVONthink as deleting 1205 documents would take a moment there too. When you had selected another database during the deletion you would not have seen what actually happened.
Had the complete database been cleaned? Do you use any scripts in Smart Rules? Can you find any of the missing files in the OS trash?
I have TimeMachine running, but it goes only 1 year in the past. In the last months, I haven’t worked quite often with DT… only added some documents regularly in the “inbox” for later processing (a bit lazy, I know) ;-). But in the backups, the groups are also missing (these were groups from 2016/2017/2018/2019). Long story short, I did not know, from which time on the groups are missing.
I have some smart rules, yes, but not for the account statements and there is never a delete-action involved.
I did a repair of the database, there was no problem.
And if there is no background job deleting/removing “old” documents or documents which hits some special criteria, then I think, I have no chance to find out, what was happening.
I will now make a screenshot any few days from the amount of documents in the database (database-statistics) and will observe this in the next weeks/months…
You’ve probably already worked through the following, and it won’t resolve your problem - but it might provide some approaches to diagnosing what’s happened. I hope it helps (and that I’ve understood your situation)
If the Time Machine backups go back 1 year
and the DTTG deletion happened very recently
Then the possible scenarios seem to be:
Your DT databases are not on a drive/in a folder that backs up to Time Machine
The deletion (or whatever) in DT on the Mac happened over a year ago AND you haven’t synced with DTTG in all that time (is that a possibility?)
I should also note these points about Time Machine:
When Time Machine does its first backup, it backs up everything on the relevant disks/folders. So your first TM backup should include any DT files that were there at the time
If the Time Machine disk fills up, it starts to delete old backups starting with the earliest, so in that case, it might be possible for old DT databases to have been lost - if they were not edited or otherwise changed in all that time
If you replaced the Time Machine disk a year ago AND your DT databases had been lost before the replacement, then they would be gone
@BLUEFROG I am always importing, never “only indexing”…
@ThatGuy The problem is, that I did not exactly know, when the folders disappear, because I mentioned the missing now, but it could be some/many times before, because I did not look at these groups for a very long time.
I checked, and the DT database is definitely on each the time-machine backup… (I use a Synology DS 718+).
I am sure, that I completely synced DTTG, when I updated to version 3 some weeks ago.
As mentioned in my last reply, I think, with the current information, a solution/finding of the problem is not possible. I will observe the topic further, check the amount iof documents stored in the database and will check, if this number DEcreases in some way.
The first important thing was, that there is no automatism for deleting documents in any way.
Thanks again for your time/help… I will update here, if I have further hints…
To minimize this kind of issues, that happened to me so and then due Dropbox and Layer 8 incompetence, I have a TOC for each database, and I’ve manually added the amount of documents for each first level Group and check that periodically. As my databases are mostly unchanged and for reference and ordered by years, it is easy and fast to check.
Another safe-rule I follow at work and at home, is check trash bin before empty it. DT has two levels of safety on deleting files: first they go to DT Bin and once removed from that place, into macOS Bin.
My dream is to have one script that will count and verify at least number of files in each group.
Said that, I’ve never lost any non-indexed file if I haven’t manually deleted.
FWIW, I’ve had a number of files go missing on multiple occasions from a large indexed folder In DEVONthink. Fortunately, I was able to restore them using Time Machine. I’m really worried about using DT moving forward to index external folders. Before version 3, I never had this issue.
It’s definitely not a sync issue as the sync doesn’t use the Finder’s trash. Do you use any smart rules or scripts which delete items or move them to DEVONthink’s trash?
Had you emptied the system Trash before you restored the missing files so it would be clear you subsequent actions were involved? If, I suggest you do.
How exactly did you do that? I ask, because it should not be possible for you to open both the backup and the current version of the database at the same time; both databases must have the same id.
Yes I did… this is NO one time issue… It keep happening not matter what I do…
I did a database repair - no errors
Rebuild - OK
close and open the database - same