Indexing folders on multiple machines creating copies of files?

rmschne · March 30, 2022, 4:53pm

I’ll take that bet!

Nope. See below. Only OneDrive there, and I did a ls to show even hidden files. And if it were an alias wouldn’t it say so in the “ls” or finder info screens?

((base) iMac:cloudstorage rmschne$ pwd
/Users/rmschne/library/cloudstorage
(base) iMac:cloudstorage rmschne$ ls -la
total 24
drwxr-xr-x+   4 rmschne  staff   128  3 Feb 09:30 .
drwx------@ 134 rmschne  staff  4288  4 Feb 13:13 ..
-rw-r--r--@   1 rmschne  staff  8196 11 Mar 18:05 .DS_Store
drwx------@  10 rmschne  staff   320 30 Mar 08:31 OneDrive-Personal

Repeating what I observed, works fine if DEVONthink does not sync this indexed folder that is a Google Drive Folder and instead letting Google Drive take care of all synching, and let DEVONthink take care of noting changes in the indexed folders without synching. both machines involved show correct files/folders in DEVONthink with this approach; whereas, with DEVONthink synch also turned on things got messed up–files moved to DEVONthink trash, duplicate folders, etc. (as OP reported). I’ve not tested with any other sync service.

(base) iMac:cloudstorage rmschne$ pwd
/Users/rmschne/library/cloudstorage
(base) iMac:cloudstorage rmschne$ ls -la
total 24
drwxr-xr-x+   4 rmschne  staff   128  3 Feb 09:30 .
drwx------@ 134 rmschne  staff  4288  4 Feb 13:13 ..
-rw-r--r--@   1 rmschne  staff  8196 11 Mar 18:05 .DS_Store
drwx------@  10 rmschne  staff   320 30 Mar 08:31 OneDrive-Personal

Can’t comment. They all have their own way of doing things. I have not experimented with these other sync services. But something going on with Google Drive when both Google Drive sync and DEVONthink sync for that index folder are turned on.

Blanc · March 30, 2022, 4:55pm

Thanks for delving into this to satisfy my appetite for knowledge. Here, this is yours

rmschne · March 30, 2022, 5:19pm

I assume digital money behind that digital image … will go well at the pub this evening.

rmschne · March 30, 2022, 5:24pm

I guess conceptually this now all making sense to me. DEVONthink indexing is to give a view into the outside world beyond DEVONthink where DEVONthink has no real control but a “view” and some strings. The outside world managed by outside forces (Google Drive), so let it do its thing and not expect DEVONthink to be doing it.

Blanc · March 30, 2022, 5:26pm

I agree, but how on earth did it ever work? enjoy your digital

rmschne · March 30, 2022, 5:42pm

Something changed. And now it sort of works with mess ups.

BLUEFROG · March 30, 2022, 8:54pm

What do you see in ~/Library/Application Support/Google/DriveFS ?

What is the yield from these Terminal commands…

defaults read /Library/Preferences/com.google.drivefs.settings ContentCachePath
defaults read /Library/Preferences/com.google.drivefs.settings DefaultMountPoint

rmschne · March 30, 2022, 9:12pm

here you go. Hope this means something to you. Gobbledygook to me.

Like I said, works great if I don’t attempt to use DEVONthink sync on these folders in addition to Google Drive’s sync, for what it’s worth.

(base) iMac:DriveFs rmschne$ pwd
/Users/rmschne/library/Application Support/Google/DriveFs
(base) iMac:DriveFs rmschne$ ls -la
total 320
drwx------   14 rmschne  staff     448 30 Mar 16:51 .
drwx------    9 rmschne  staff     288 10 Sep  2021 ..
-rw-r--r--@   1 rmschne  staff    6148 17 Mar 13:17 .DS_Store
drwx------@  23 rmschne  staff     736 30 Mar 16:51 116680860864544195952
drwx------@   7 rmschne  staff     224 10 Sep  2021 Crashpad
drwx------  303 rmschne  staff    9696 30 Mar 16:14 Logs
drwx------  338 rmschne  staff   10816  7 Feb 15:34 Resources
drwx------   21 rmschne  staff     672 30 Mar 12:52 cef_cache
-rw-------    1 rmschne  staff     356  8 Mar 18:30 global_feature_config
-rw-------    1 rmschne  staff   12288 30 Mar 16:51 metrics_store_sqlite.db
drwx------    5 rmschne  staff     160 10 Sep  2021 migration
-rw-------    1 rmschne  staff      61 10 Sep  2021 migration_data
-rw-------    1 rmschne  staff       3 22 Mar 08:25 pid.txt
-rw-------    1 rmschne  staff  131072 30 Mar 16:51 root_preference_sqlite.db

and

(base) iMac:DriveFs rmschne$ defaults read /Library/Preferences/com.google.drivefs.settings ContentCachePath
2022-03-30 22:09:37.584 defaults[25832:5956810] 
The domain/default pair of (/Library/Preferences/com.google.drivefs.settings, ContentCachePath) does not exist

and

(base) iMac:DriveFs rmschne$ defaults read /Library/Preferences/com.google.drivefs.settings DefaultMountPoint
2022-03-30 22:10:34.859 defaults[25875:5957499] 
The domain/default pair of (/Library/Preferences/com.google.drivefs.settings, DefaultMountPoint) does not exist

BLUEFROG · March 30, 2022, 9:26pm

Thanks.

Can you try the User library on the defaults commands?

mikebore · March 30, 2022, 9:35pm

Interesting discussion thanks.

I understand the basic logic but, as Blanc said, not why it doesn’t apply to Dropbox, OneDrive and iCloud indexed locations. Is there something known to be different about Google drive.

I have all my files on Dropbox and indexed in DT3. All on two computers where Dropbox and DT are both syncing.

I am seeing occasional file duplicates which maybe caused by the same mechanism but not apparently on your scale. My dupes have “copy” added to filename…I guess this may just be the difference between how Google and Dropbox label dupes.

rmschne · March 31, 2022, 5:43am

Here you go …

(base) iMac:~ rmschne$ defaults read ~/Library/Preferences/com.google.drivefs.settings ContentCachePath
2022-03-31 06:41:44.148 defaults[39820:6125158] 
The domain/default pair of (/Users/rmschne/Library/Preferences/com.google.drivefs.settings, ContentCachePath) does not exist
(base) iMac:~ rmschne$ defaults read ~/Library/Preferences/com.google.drivefs.settings DefaultMountPoint
2022-03-31 06:42:10.941 defaults[39843:6125426] 
The domain/default pair of (/Users/rmschne/Library/Preferences/com.google.drivefs.settings, DefaultMountPoint) does not exist

rmschne · March 31, 2022, 5:47am

I don’t know. I’m currently not using DEVONthink to index into any other cloud-synced folders. I used to until Dropbox’s subscription fees got so high to cause me to migrate away but for the “free” space they offer. As I get time (and at the moment I have a couple of other projects underway so time short) not sure when I can repeat the experiment. One of my projects involves files in a shared folder in Google Drive, hence I noticed the possible issues (unless I’m being fooled … easy to do!).

cgrunenberg · March 31, 2022, 7:31am

The indexed groups or the folders in the filesystem were duplicated? And was the path of indexed groups the same one or different? So far the most likely reason to me still seems to be that the paths on all your Macs are not identical.

rmschne · March 31, 2022, 7:55am

Path names identical. I’m positive.

After both Google Drive and Dropbox syncing active (they are both going at their own pace without my intervention) I watched folders being duplicated, folders not showing all files, DEVONthink moving files it thinks are no longer in the indexed folder into trash.

At the moment I have turned off DEVONthink sync for that database which holds only one index folder (and subfolders) that resides in the Google Drive and relying on Google Drive to sync the two machines, then DEVONthink updates index and all is well. When DEVONthink syncs it messes things up and I think that due to indeterminate state of Google Drive sync. I think you said above DEVONthink recognises index pointing to cloud folder. Perhaps a change somewhere caused this detection to fail. I know I am amateurishly grasping for what happens in all the software products involved without any real knowledge of that.

At this point can I suggest you try? I think I described my experiment setup above. If you need another, clearer, step by step experiment explanation, let me know.

rhaynes74 · March 31, 2022, 10:21am

Hi folks, so do we have a conclusion? I would like to have the ability to add or annotate files in DT on either machine. I would also like to have the GD updated (which would allow me to access these files should I not have access to DT).

rmschne · March 31, 2022, 11:10am

I recommend you try and do your own experiment.

BLUEFROG · March 31, 2022, 12:23pm

What is the path of an indexed file and its copy as shown in DEVONthink’s Generic Info inspector?

rmschne · March 31, 2022, 12:48pm

I assume you are asking me and not @rhaynes74 and still searching for aliases and file paths not exactly same on each machine? Far as I know, there are no aliases and file paths are exactly same.

With DEVONthink syncing for this database turned OFF and relying only on Google Drive app to do the synching across two devices, there are no duplicate folders or files and no files (far as I can see) have been moved to DEVONthink trash because I’m running it a way which I think makes everything work fine. I haven’t tried to break it again by turning on DEVONthink Sync in addition to Google Sync. As I think this sets up an indeterminate state with variables about what is in the indexed folders before, during, and after Google Drive sync.

An example file:

in the Google Drive

"Macintosh HD/users/rmschne/google drive/family/rhs/test_folder/test.docx"

in DEVONthink Info for that file which has it index updated says path is

"~/Google Drive/Family/RHS/test_folder/test.docx"

(DEVONthink’s Info Path field clips off right end, though. text string length, I guess.)

BLUEFROG · March 31, 2022, 1:17pm

Thanks!
Can you post the Get Info pane for that file in the Finder?

rmschne · March 31, 2022, 1:32pm

Still wondering if barking up wrong tree, but I defer to the Great One!

Two screen shots. One from the iMac and one from the Macbook.