Duplicated extensions - indexed folders

I am running DEVONThink Pro 3.0 and have noticed that in my indexed folders about half of the files have had their extensions duplicated. For example Test.pdf become Test.pdf.pdf and Test.jpg become Test.jpg.jpeg (note that for JPEG files the extension is changed from jpg to jpeg). I can not work out why some files have had their extensions duplicated and some have not. I did have ‘show extensions’ turned off in preferences.

Did you organize/rename the files in DEVONthink 2.7.2?

No, i have been using DEVONthink 3 exclusively throughout the beta programme

So, you are not syncing the database to DEVONthink To Go?

Where are your indexed folders located?

No I am syncing the database to DTTG via iCloud sync. The files are located in the Finder, they are not in any other cloud system (apart from the sync location described above). I have a seperate issue with one of the indexed folders in DTTG (not sure that it is related but will state it anyway) and that is that groups, within the indexed folder, that are deleted in DEVONthink Pro are not removed from DTTG even after I empty the trash.

I have a seperate issue with one of the indexed folders in DTTG (not sure that it is related but will state it anyway) and that is that groups, within the indexed folder, that are deleted in DEVONthink Pro are not removed from DTTG even after I empty the trash

I am unable tor reproduce this issue here.

  1. An indexed folder with subfolders, indexed into DEVONthink 3.
  2. Synced to DEVONthink To Go. All groups accounted for.
  3. Delete subgroup in DEVONthink and empty the Trash.
    Do note: Emptying the database’s Trash, moves the subfolder in the Finder to the system Trash.
  4. Sync to DEVONthink To Go and the folder is gone.

I have attached some screenshots to illustrate problem 2 (not the original problem that I posted about which is the duplicated extensions as I have manually changed the names). The first shows 1033 files in my ‘References’ group that is an indexed folder:


The second shows the first few files within this group ordered by name:

Third shows same database in DTTG with more items in it than DEVONthink Pro 3:

And the final screenshot show the first few contents of this group ordered by name showing empty groups (and 1 group with a file in it) that are not present in DEVONthink Pro 3

Well, was a strange one to troubleshoot… :sweat_smile:

I have/had the exact same issue as @kseggleton . Almost identical setup: Indexed folders, synced with DTTG via iCloud, “.pdf” became “.pdf.pdf”, “.jpg” became “.jpg.jpeg”. “Show extensions” is turned off. I also made a clean break from DTPO 2 to DTP 3 – stopped using the former, started using the latter. (When I turned off sync in DTPO 2 and turned it on in DTP 3, I noticed that a substantial number of files were sync’ed again, but somehow it didn’t quite click what had happened.)

Not every file is affected somehow: In a folder of bank statements, about 90-95% were affected, while the other ones still had their original extensions. I also noticed that only folders with more than ~10 files were affected… as I said, the whole thing is rather strange.

I had missed the warning that DEVONthink shouldn’t be used to index cloud folders by other services: The affected folders are synced with Nextcloud. I would like to change this now: What is the recommended procedure to move folders indexed by DTP 3 to a different location, ideally preserving the existing UUIDs?
(EDIT: I just noticed that two of the affected folders were not actually in the NextCloud folders. They were only indexed by DT. Both of them contained a large number of files. This really is a strange issue…)

I noticed this issue today when I tried to use one of my databases, which is an indexed Calibre library. In short, some rather bizarre stuff happens: Calibre somehow recreates the file with a single extension, this gets indexed by DT and suddenly my database was getting duplicates for every book I clicked on in Calibre…

I am now cleaning up my databases. I figured this might be a decent opportunity to get to know the new “smart rules” feature. I am sharing this in the hopes that this might be useful for others.

Create a smart rule with the following conditions

  • file name ends in “.pdf.pdf” (change this condition as you go through the different file types)
  • object is not duplicate (shouldn’t affect non-Calibre users)
  • object is indexed

I wrote the following AppleScript as an action for the smart rule.

Caveats:

  1. It seems to do the trick, but I am a horrible AppleScript writer – use at your own risk.
  2. The script only knows how to handle file extensions with three or four characters.
  3. Also,DEVONthink afterwards treats these files as having been modified, while the macOS Finder doesn’t.

The AppleScript

on performSmartRule(theRecords)
tell application id "DNtp"
	activate
	
	repeat with theRecord in theRecords
		set filePath to ((path of theRecord) as string)
		set HFSfile to POSIX file filePath as alias
		
		set theFileName to (filename of theRecord as string)
		
		set theExt to characters -4 through -1 of theFileName as string
		
		if theExt contains "." then
			set extOffset to -5
		else
			set extOffset to -6
		end if
		
		set newName to (characters 1 through extOffset of theFileName) as string
		
		set newPath to (characters 1 through extOffset of filePath as string)
		set path of theRecord to newPath
		
		tell application "Finder"
			set FinderRecord to file HFSfile
			set the name of FinderRecord to newName
		end tell
		
	end repeat
end tell
end performSmartRule

For what it’s worth, I have the exact same issue. I have a “notes” folder in DropBox and tons of files that ended up .txt.txt and .md.md and .pdf.pdf. But the issue does NOT seem to be limited to folders indexed off of DropBox. It’s just more prevalent there as those tend to be “collections” with large numbers of files.