Adding empty files to a database breaks sync silently

Thanks :slight_smile: as an aside, I can’t get type of theRecord is not smart group to work (the term smart group is not recognised); type of theRecord is not group does as it should. Admittedly, “smart group” is not listed as a type in the dictionary; but apparently a “smart group” is not a “group”, either.

Is there any way to exclude a type smart group in an if/then?

There wasn’t a ‘triangle’ symbol warning you about a sync issue at the bottom in the sync log?

The Log panel should be automatically shown on the Mac (if this option isn’t disabled).

I’m currently facing the same problem. However, I have two databases with indexed folders. Those folders contain empty files required by Marked.app (<name-xyz>.marked). Now the log panel pops up automatically and displays every minute a message that the db check failed and I should repair the db.

So, an option to toggle this check would be highly appreciated.

2 Likes

Thank you for providing context, background and possible solutions. This really helped me understand the reasons for what’s going on.

Do you mean the Sync tab in Preferences? This is not a place that I go to very often, but no I don’t see any warnings here. The only indication I got that something was wrong with the sync was that I noticed that the date of the last sync was two weeks ago. A clearer warning or error message would have been great.

I have this disabled. I will consider enabling it, but I don’t care much about general log messages and floating windows. I would prefer some kind of clearly visible yellow or red warning sign for fatal errors such as failed syncs. That would be fantastic.

1 Like

No, top right hand corner of your DT window, or on the cloud symbol in DTTG

I removed the log triangle from my menu, but adding it back the only indication I get is that there is a new log message:

image

I probably wouldn’t think to check this as I had no idea that something was wrong with the sync before I went on the road and noticed missing notes on my iPhone. So again, I would love to have some kind of log level indicator somewhere in the UI to let me know not just when I have new log messages, but when I have critical/fatal errors in the log specifically.

I realize this is totally in feature request territory now, and I don’t expect anyone to promise anything here. I’m really happy to have been provided some context and I’m also fairly confident I will be able to prevent this issue from reappearing now that I know the cause of the failed syncs and suggestions to mitigate it. So thanks for that! :slight_smile:

1 Like

You’re most welcome :slight_smile: I’ve actually taken to a brief daily look at the :warning: - some of my daily scripts report any errors to the log, so it’s become part of my routine :slight_smile:

I should perhaps also note that I also got no indication in DTTG that anything was wrong. I did have some weird repeated syncs happening, but nothing that was indicated as errors in the UI.

I will definitely make this a habit from now on! Thanks for the tip. :slight_smile:

1 Like

+1 to fixing this – I have a situation where I cannot delete the empty file (indexing a read-only source).

1 Like

As a temporary fix, can you put that index in a separate database? Then your other databases would (presumably) still sync.

1 Like

Yes that’s a reasonable short-term solution - thanks

Now that that’s working… :slight_smile:

It does return the critical database to syncing across my devices, but a huge benefit was being able to access that indexed r/o directory on mobile devices, so it’s definitely only “partial restoration of service”.

Seems like a clear and critical bug – much appreciated for taking it seriously and hopefully we can have a fix soon!

  • Eric

It’s actually not a bug. It’s working as intended. However, we are looking at some modifications that could allow for some unanticipated use cases.

1 Like

An empty file is a legal file type on MacOS and iOS. The fact that there is a kind of legal file which, when present, prevents sync from working seems to me indistinguishable from a bug. Are there other instances of legal files that stop sync from working? That also seems like a security risk - if somebody can share a file the contents of which interfere with how sync works, that’s a scary attack vector.

Whether it is a code bug or a design bug is a different question.

1 Like

The sync intentionally stops to prevent possible damage to files in certain circumstances:

The point that the OS allows zero-byte files isn’t the issue. The issue is there can be zero-byte files that are problematic in a database, hence the sync is inhibited to avoid passing along any inconsistencies.

As I said, we are considering some modifications to allow for certain use cases.

Is it fair to say there was a bug in DT which led to data loss (specifically by setting files in certain circumstances to zero-length and then replicating those modifications), and that the fix was to disallow replication of zero-length files? And that this has the side-effect that DT doesn’t work correctly for people whose databases contain legitimate zero-length files?

If that’s an accurate summary of the situation, I would have preferred more robust communication about the issue from the start – aren’t we now in the situation where we may have lost data and we may not be aware of it, and we may even have gone and deleted the zero-length files without first taking the time to understand what data may have been lost?

Maybe you can remind us what date this bug showed up so we can explore the possibility of needing to restore from backup.

Is there a way to “diff” between two versions of a database?

See Update on potential data loss