DT3 import stopped

DT 3.8.6 (17457)

The initial problem of the error described below was the lack of .manifest files “in” the sync database.
These errors are apparently caused by faulty synchronization of various devices (laptop, iPhone, iPad, etc.) and I was unable to solve them despite searching the forums (why hasn’t this problem been solved for years ?).

Therefore, I erased the entire sync database and rebuilt it. The “master” was an iMac, where the latest version was definitely saved.
An iPhone was then synchronized without any problems and then an iPad.

However, massive storage problems occured here:
128 GB are installed on the iPad
58 GB size of sync database (du -hs)
68 GB size of the original DB (du -hs)
→ so enough size availible ?

Nevertheless, the import of the data constantly stopped during the sync !
According to “iPad storage” there are still 20 GB free.

DT3 obviously creates extremely large temporary files during import and indexing. However, it is no longer possible to understand why this iPad can no longer be used for DT3.


How can the problem be solved now ?

What sync method are you using?
Bonjour does not use a sync database

1 Like

sync-server connected via WebDAV

I’m not sure what you mean by the term “sync-server” AFAIK it’s not a term used in the DEVONthink world … however,

could you please post a screen shot of the WebDAV settings and also please indicate the WebDAV server “brand”, e.g. Synology or an internet service?

I’m also wondering if there is sufficient space on the iPad … 20 gb might not be enough given the size of your databases but for this @bluefrog or @cgrunenberg need to step in.

I sync many databases to my iPad and iPhone and I’m also using WebDav.

One of the settings on the iPhone and iPad is to specify how many items should be kept on the device (The others are downloaded on demand). I have this set to 250 items. I’m not certain about this but maybe that will help limit the size of your databases and temp files. It is worth a try.

1 Like

in our case it make no sence because the last limit is 1.000 (or all) and we often look at different and large directories, so that reloading has to be done again and again.
Since the iPads are often used abroad, reloading would be done constantly.

Regardless of this: statistically, there are still 20GB free. Why does the sync mechanism stop working beforehand ?

While waiting for the WebDAV questions to be addressed, perhaps temporarily turn WebDAV sync off and try Bonjour for the iOS devices. May well work better. Yes, local network only, but may work on iOS devices with not much memory left. Just a hunch that your WebDAV is local network also, so local network may not be a factor. Just guessing as you have not said.

@rmschne

I’m not sure what you mean by the term “sync-server” AFAIK it’s not a term used in the DEVONthink world … however,

server hwo contain the sync data (= WebDAV server).

WebDAV-Server:
Samba, 4.18.3
Apache2, /2.4.59
Debian 5.10.218-1~deb10u1
running extremly stable - same as internal network.

I’m also wondering if there is sufficient space on the iPad … 20 gb might not be enough given the size of your databases but for this @bluefrog or @cgrunenberg need to step in.

20 GB is the free space, after DT3 import stopped ! At this point DT3 used 66,09 GB for prog & data.

Is there anything in /var/log/apache on your server (either access.log or error.log) that might indicate a problem?

I have 1.9 GB on my Mac, 2.0 GB on the WebDAV server’s sync store (Synology)
On the iPad, that translates to 2.4 GB. Which is about 25 percent more than on the Mac/server.
In your case, that could mean (68 + 25%) 85 GB on the iPad. Which is an additional 17 GB – close to filling the device up.

@chrillek

in error.log only for the iPad in the past

[Sat Aug 24 08:47:51.652194 2024] [proxy_http:error] [pid 25692] (70007)The timeout specified has expired: [client 192.168.x.xxx:63659] AH01110: error reading response

and access.log

192.168.x.x - user [19/Aug/2024:23:58:56 +0200] “PROPFIND /webdav/XXXXX.dtCloud/xxxxx88da1abed249cc2a98ef52807e4ed2f5aa0b07837918b78dbfdfbb218e2/745742459.023-c3b4c84252b87ee5db84b5f4f56fcef258c9a140-25586xxxxx.manifest HTTP/1.1” 404 537 “-” “DEVONcloudy 1.30.6”

both messagesoccurs very frequently.

On the iPad, that translates to 2.4 GB. Which is about 25 percent more than on the Mac/server.
In your case, that could mean (68 + 25%) 85 GB on the iPad. Which is an additional 17 GB – close to filling the device up.

You’re right. It’s going to be very tight.
The question is why DT3 import requires 25% more storage space on mobile devices when these devices have much less storage space than desktops ?

The space requirements have been discussed and explained here previously. A forum search should tell you more.

@rmschne

Just a hunch that your WebDAV is local network also, so local network may not be a factor.

I’m currently testing Bonjour, with an iMac as the server.
I’m just surprised that the sync is done from the local (remaining DB) on the iPad to the iMac.
I would have expected it to be the other way around: from the iMac to the iPad ?

It’s a two-way sync, so perhaps there are things on the iPad not on the iMac.

If not done already, ensure you enable Bonjour only on the iMac as you consider that the “server”.

It’s a two-way sync, so perhaps there are things on the iPad not on the iMac.

yes, right…
… but the DB on the iPad was the “leftovers” from the aborted import from the WebDAV server.
It is therefore unclear whether the iPad synced this (remaining) data to the iMac. According to the counter, it has synced all the data imported up to that point (> 45,000 files).

If not done already, ensure you enable Bonjour only on the iMac as you consider that the “server”.

it is only enabled on the iMac (= Bonjour server)

I guess if I had this uncertainty about things as you seem to have, I’d start again on the iOS devices. Remove or disable the WebDav sync setup (perhaps temporarily), delete the databases on the iOS devices, re-start iOS Devices (with “hope” that the re-start would clean up space released by DEVONthink ToGo), and then use Bonjour to do a fresh sync without any consideration of “leftovers”.