Fail to restore cleared items in indexed groups

I reach a serious impasse with my DTP workflow:

  1. I have a huge academic papers library kept in a Finder 15GB folder and get indexed in DevonThinkPro, say it A group;
  2. Playing around in DTP I send to trash (intentionally or not) a file reference in A;
  3. That’s OK: my file is intact in Finder but it has be cleared from A in DTP;
  4. Now it happens that I want that file back in DTP A;
  5. I click on File/Update Indexed Items menu (ex Sync command);
  6. Ooops!.. The offended file doesn’t reappear in A, as I expect :blush: ;
  7. Now I have to remember something about the file (was it wkdk3484i.pdf or 123jfd45.pdf?) manually search it in Finder and re-index the file in DTP… etc. etc.

Is this a trusty system?

The problem is specially serious when working with replicants: when, after a research, I want to clean various replicants in their locations, it’s too easy to also delete the original file and so to loose track of it. Nor DTP doesn’t seem to automatically (or semi-automatically) repair modified or outdated links as other apps yes do (vide Omnifocus).

I would prefer:

  1. to keep my original files always in their original location, a big library in Folder indexed in DTP;
  2. to freely play around (copy, move, delete, etc.) with replicants in DTP;
  3. to simply sync/restore/repair/clean the original location getting all the files re-indexes in DTP as they actually are in the Finder. I don’t want a (everlasting and CPU fogging) re-indexing of the folder every time I realize something has be deleted, only quickly fix the relevant changes and restore the indexing links;
  4. not to worry about the integrity between Finder and DTP indexed locations, but simply and cleanly trust on DTP: IMHO, DTP has to be aware what actually there is and there is not in Finder, not me.

Any hint?

Trusty? Yes, as that is the intended behavior. If you have deleted files in the database that were contained in indexed groups, DEVONthink records marks the record of the files accordingly and will not re-index them with the Update Indexed Items (Synchronize) command. Think a moment about the alternative here-say you have intentionally deleted hundreds of documents that were indexed from the 15GB Finder folder as you never want them to appear in the database. Every time you run the Update Indexed Items command, these documents would be re-indexed into the database. This behavior would make the Index function worthless to many users.

Replicants, and duplicates, are marked as such, either by color or by an icon (depending on the preference setting) so I always know if I am deleting the last remaining instance of the document. When documents are deleted, they are placed in the Trash folder of the database. As you are wanting to index all your documents, you might want to experiment with leaving them in the Trash rather than deleting them. Unlike imported documents in the database, the indexed documents in the Trash do not take up any additional space in the database/on disk.

Yes, I agree completely with this point. I too would like the ability to re-link indexed folders, even if just by editing the path in the Info pane.

Thanks for the hints!

I agree only partially with that… Maybe it would be better to give the user the very last decision. Proposals:

  1. a last minute re-indexing polite warning dialog (“xxx.hundreds previously cleared documents will be reindex in the group if you click OK… Are you really really sure you want to do that, my friend?”);
  2. more professional and useful: a smart panel where it would be possible to select/deselect the individual files.

That’s interesting, I didn’t know it. I’ve always seen Trash quite a trivial step before getting rid of stuff. I’ll see it now with more respect. :slight_smile:

Good, let us push for the re-link new great function!

[Wish for another great feature: going back and forward along the navigation history on a window basis, not only previous viewed document… maybe in other thread]

Well, I’ve experimented with this a little more this morning, and either the behavior for synchronizing has changed, or my memory of how it worked is bad (most likely option). My apologies for giving the wrong information.

What I am seeing with DEVONthink Pro Office 2.3 is this-if an indexed document is moved from the indexed group, or if an indexed document is replicated to another group then the original is deleted from the indexed group, then the Update Indexed Items command will not update the indexed group with another instance of the document. This makes sense-if you want to index the document and then move it to a different group, then it should not reappear in the indexed group when synced.

However, if all instances of the indexed document have been deleted from the database, the running the Update Indexed Items command will recreate the document in the Indexed group. Is this the behavior that you are wanting, or does it not work this way for you?

Let examine a real case:

  • I have a group A in DTP with 4 files that indexes a folder A in Finder;
  • I create a group B in DTP;
  • I replicate one file of A to B: I see now a red files both in A and in B.
  • Here is the problem: I have now two visually identical files in A and in B, but the one in A is the real indexed file and the one in B is its replicant: lack of visual hint;
  • Let’s clear the replicant in B; the file in A turns black… OK.
  • Let’s keep instead the replicant in B and clear the original file (but seeming a replicant one) in A: the file turns black in B (i.e. it seems an original one), but it is now really in the trash, but I keep seeing it in B (no visual hint, once more).
  • Now I want my file got back in A to reestablish the link integrity and cut the mess: if I do an “Update indexed items” (or “Synchronize”) I’d like to get exactly that, i.e. my indexed items updated (well, re-synched with the Finder folder), no an “Update indexed item not previously cleared from index update”, that is exactly what DTP has decided to do. This is not my meaning of sync or update concept. If I decide to have a Finder folder synced to DTP, I mean this: what it’s seen in Finder is exactly mirrored in DTP. If I want to mess with files, I’ll create another group and use replicants or duplicated.

I insist that there are odd behaviors and quite arbitrary too from the program:

  1. replicants and originals get the same identical appearance (inducing in mistakes): it’s wonderful to play and messing with replicants and duplicated, but the original source has to remain always untouched, visually more prominent, and immediately recoverable after being cleared if the user wants to;
  2. the file actually in the trash keeps appearing in black (like an original one) inside the group from where it has been removed: I have to be careful about a little tiny gray text string above the menu bar to know that it is really in the trash. Unfriendly: I would prefer a dimmed appearance for files in the trash.
  3. when I have red files (replicants AND originals) or blue file (duplicated AND original), I have no way to know which are what, or which descends from what, that fights against a basic order principle.
  4. nor I have an easy way to know where are the instances located: I have to open the inspector, click on the instances button and getting an odd information in a remote pop-up panel, that’s all. Unfriendly.
  5. this and the lack of navigation history (like that in the Finder) leads me to loose myself too many times.

Proposals for DTP developers:

  1. IMHO the replicant/duplicate feature has to be better visually implemented and
  2. the “Update Indexed Items” command has to give the user the last decision about restoring link AND files.

I’ve tried this, but If I clear all instances of the indexed document the Update Indexed Items command will NOT recreate ANY document at all in the Indexed group. I have to re-index the Finder folder if I want the documents in DTP or to recover them from the Trash.

About the Trash: it can be an option, I don’t say it isn’t… but what about recovering that forgotten individual file from the trash after two weeks working in other 7 projects and messing around with hundreds of files, replicants, duplicates, groups, tags, etc.? There is only one place where I’m sure the file is there: the Finder. So a click on “Update Indexed Items” should be like the million cleaning/order resetting tip.

It will be helpful to understand that (*with one exception) all replicants are identical and equal. If you have your scenario where you replicate from A to B, and then delete the “original” in A, the replicant in B is still linked to the indexed folder in the Finder. For this discussion, let us assume that the document is a text file. If you open that document from the Finder folder A in TextEdit, make and save changes, the changes will be reflected in the document in Group B. Likewise, if you edit the text file in Group B in DEVONthink, save the changes, and then open the text file from the Finder folder A in TextEdit, you’ll see the changes that you made in DEVONthink. Replicants in DEVONthink do not behave the same as originals and aliases in the Finder.

  • The exception is that if you do a Global search in DEVONthink, it will the first instance of the replicant, not all instances of the replicant.

That is what I too thought when I first posted in this thread, but my test today confirmed that the documents will be reindexed in the database if all replicants have been deleted from the database, and the trash emptied.

It might be helpful to try this-create a new, empty group “Test” and drag all your groups into this group, except for your indexed Group A. Then create a smart group that looks like this:

This will return all indexed documents that are not in the indexed Group A. Does this find any of the documents that are not appearing with the Update Indexed Items command that you are executing on Group A?

Yes, that is! The step I was missing was empty the Trash. It sound quite like a “please, switch off the toaster to get your cat sleeping” tip (really DTP has some odd things like that), but finally it works. After emptying the Trash, the Update Indexed Items works as I expect: the cleared items jump back to the indexing group and the link integrity seems reset.

Thank you very much, Greg!

Glad to help. :wink: