Database inconsistencies after upgrade to DT3.6.3

Dear all,

I just upgraded to DT 3.6.3 and one of my databases now shows 4 inconsistencies. I am still running all databases on legacy sync with iCloud. Verify & Repair as well as Rebuild of the database did not resolve the issue.
Is this just a coincidence (upgrade and inconsistencies happening at the same time) and I need to retrieve my Timemachine backups or is there more to it?

Thanks

Hi :slight_smile:

I’ll wager it’s a coincidence [Edit: yes and no; see this post further down]. Please could you be more precise what you mean by inconsistencies; preferably post a screenshot of any error message. That will make it easier for anybody here to assist you.

Thanks for your reply. When doing the “Verify & Repair” on the database, a window with the message “Verification databaseName: Found 4 inconsistencies, 0 invalid filenames, 0 missing and 0 orphaned files” comes up, clicking on “Repair” brings “Repair failed 4 errors left”.
Is there any other location I can look for more information?

It might be worth emptying trash and restarting your Mac if you have not done so. I’m surprised Rebuild didn’t solve this, so @BLUEFROG might have some more helpful input for you.

I think I managed to resolve the issue. The log file (see screenshot) showed 4 folders with empty files. I deleted the folders from DT, emptied the trash and verified again. The database is consistent again (ufff…).
Three of troublemaker folders are of the same basic structure, actually there are many more of them that did not cause trouble. There is one difference nevertheless, the three problematic folders contained one pdf file each with the name given in the log file. I have no idea how the empty pdf files came into the folder.

Here is the screenshot from the log file.

Ok, so far so good. Do you use DEVONthink to go, and if so, have you updated to version 3, and if so, did you update prior to version 3.0.3 being published and if so, did you sync with your Mac?

My answers are in your comment.

Ok, so far so good. Do you use DEVONthink to go YES, and if so,
have you updated to version 3 YES, and if so,
did you update prior to version 3.0.3 being published YES and if so,
did you sync with your Mac YES?
But so far I did not encounter any problems, I have just updated the iOS devices to the most recent version, and synced test files going both ways (DT to DTTG and DTTG to DT), it works fine.
Furthermore, the 4th problematic folder contained an empty text file, maybe empty files cause the problem?

One more remark, I checked the empty files, their creation date and time is similar to those of the other files in the same folder. So I assume that they were there all along.

There was a problem in the initial implementation of DTTG 3 which could - for reasons not known with certainty, but based on current assumptions possibly related to a bug previously present in a past 2.x version, cause files to lose their content (i.e., go 0-byte in size). These files have been dubbed “ghost files”; a search for that term will bring up numerous reports here in the forum. It is conceivable that that is what happened here; it is also conceivable that instead a mechanism for detecting such files has triggered, although those files were perhaps previously empty.

The good news is that - as far a I know (and I think I am up to date on DEVONtech’s feedback to users on this) - once this has happened, it is not expected to happen again. So if you have solved the problem once, it should not recur. Whilst an automatic restore was implemented in current versions of DTTG 3, it did not work for every user; if you have, in fact, lost data, you’d probably have to restore it from a backup (if it’s lots and/or you don’t have a backup, there may be an alternative route via using DTTG 2; search for “ghost files”). Just to be sure, you might want to implement a smart group in DT3 which searches for kind is any document and size is 0. Also, in DTTG3 there is a smart group called “Ghosts”; it would show up any (other) ghost files (again, if that actually was even the problem here).

1 Like

Hello I have excatly the same problem except I have 24 inconsistencies.
2 issues:
-Some files are Endnote library - this I can deal with. I know EndNote and DT don’t sympatize too well
-It look that some of my pdf got split and are not usable anymore.

I just realised I can salvage pdf files by putting them in the finder and renaming them. But what a loss of time. Is there another way…Worrysome :roll_eyes

NB:
I do not use DTTG nor any cloud service with DT

Capture d’écran, le 2021-03-06 à 12.07.09

Capture d’écran, le 2021-03-06 à 12.09.22

@cgrunenberg coincidence or a mechanism for detecting ghost files acting up maybe? The last poster at least does not fulfil the criteria for even having ghost files as far as I can tell.

A ghost file isn’t necessarily from DEVONthink To Go or sync. With a few file formats filtered out as acceptable, zero-byte files are detected and reported as problematic.

That’s what I was guessing; so these files may have been there for aeons and are only being picked up now because a new mechanism for alerting when ghost files turn up has been implemented in DT3.0.x, right? That would mean that a small influx of people suddenly reporting problems does not necessarily point to a common source of the problem, just to a common mechanism of the reporting?

Correct. The maintenance routines have been improved and this is one byproduct.

1 Like

Excellent, thanks. Whilst that still leaves the three people who have reported problems in the last 24 hours with their problems (and we’ll all tune in to help), it does mean its not a systemic worry :+1:

Please can you explain in more detail what you mean by that? The PDF files in question, are they simply dormant in DT, or are they actively used by other software too? I’ve not heard of DT splitting PDFs into parts (it does have a split function, but that does not leave PDFs with .part extensions) so I’m just wondering whether this has anything to do with DT at all. The fact that DT is now drawing your attention to the problem does not infer that the problem cropped up recently (see the posts by Jim and myself above this one).

Hi Thanks for your replies.

About the splitted files:

In the log, among the 24 inconsistencies, I had several pdf that were reported as Empty File.
Ex: Schmitt_et_coll_2000.pdf

This file was dormant in DT. I had been used it for a medical expertise a few years ago and I probably had the file standing on my finder then I dragued it into a folder in specific patient folder in DT

When the check of the DB was made yesterday, the log was stating that it was now an empty file (before the last picture just up)

I went to locate it and in fact, there were 2 files: one of 0k and another (finissing by .part) of 370 k. None of these file were readable.

I tried to rename in with .pdf instead of .part - but it didnt work in DT
So I draggued both on my finder and I was able to rename the file with .pdf - it became readble (normal).

Just went through time machine before installing the new version: Eureka !

You were totally right !

This splitted files already existed in DV since 2020-01-07 (yesterday I forgot to check their modification date and had emptied my trask)

Interesting. After the last upgrade, I had one database that was trashed – hundreds of files missing or unreadable; it couldn’t be repaired. Fortunately, I have automatic cloud backup and was able to recover a pristine copy, but it wasn’t a happy experience. At the time, I suspected the upgrade, but couldn’t be sure since my other databases were fine. I ran Verify & Repair, on everything and they all passed, which was a great relief!

Using Verify and Repair I was able to fix inconsistencies manually, except for one in a specific database.
For this one - the log does not report anything. Empty line

QUESTION : How could it be ? It used to work fine in this same database for repairing the other inconsistencies.

@ DC Berk - happy for you it has been solved. Sometimes things get frightfull…