What you say is exactly right - with the important caveat that you, Bluefrog, and I have all noted - Read and understand all of the documentation on indexing.
The OP suggested in his original post that [my paraphrasing] he is a basic user of DT who is very busy with lots of professional activities. I am a former coder [many years ago] and currently use DT for busy non-tech professional work - and when I tried indexing I missed some of the nuances, just like quite a number of other users who have posted about lost files.
It is indeed documented very well. If you follow the instructions exactly you will not have a problem. So everyone needs to decide if they indeed have that level of discipline to read everything on the topic and consistently follow it.
Bottom line - most of DT3 is a rock-solid bulletproof database. I have never lost any data except when I tried to index. DT3 has saved my bacon a few times with some of its features that help me to keep my data. But indexing is not in that category. It’s the one area of DT3 I think all but the uber-technically-competent should stay away from with important data.
Well, the first item in the link that you posted is from a user that deleted files in the Finder that were indexed in the database, then couldn’t understand why the files were no longer in the database. I don’t know how uber-technically-competent one needs to be to understand why that happened? There are many ways that the actions of the user can corrupt databases and/or lose data, and it behoves the user to explore how that might happen. Indexing documents doesn’t Inherently put the user’s data at risk.
I can see how that seems like a really amateurish mistake. But it could also be more nuanced.
I have certainly deleted files or even entire folders inadvertently at times. Usually I realize the problem soon enough when I notice that an important file is not there, and there is enough time for me to retrieve it from the Trash or elsewhere.
One issue with indexing however is that a file can be deleted from the Finder but still appear to be present in your daily DT3 user interface. When a user (or a wayward app) deletes a file in Finder, the corresponding indexed Group in DT may not immediately reflect that the file was deleted in Finder. So the busy user might understandably not recognize the file deletion until a time when he is out of town or has a critical need for the file or has already emptied the trash etc.
This is possibly the most common gotcha situation with indexed files but not the only one.
Don’t get me wrong- I think indexing has its place. It is particularly helpful as a temporary means of transfer to get files into DT3 - especially via Dropbox. And overall DT3 is maybe the most solid database I have ever used - it has let me down less often for sure than SQL databases that I have used. But I do stay away from indexing of mission-critical files.
Except maybe you should touch the Finder when you add files. Otherwise you can wind up with a folder which partially resides inside DT3 and partially resides in your Finder. Then if you restore one but not both from a backup, or if you move files from there elsewhere, or if a year from now you do not quite remember all these details, or some new app does something funky to your file system…
Indexing documents in DEVONthink 3 is very different than indexing documents in any prior version of DEVONthink. The integration with the Finder is much tighter now, to the point that one could say if you lose indexed files in DEVONthink today, then you would have lost them in the Finder anyway.
rkaplan, your characterization of me is correct … despite being very technical, I have enjoyed being a very basic DT3 user of five years. So far, I’ve not had to think about file integrity what-ifs in my current DT workflow because all I do is scan and move and search, all within the DT database and not indexed. Nothing in my DT DB changes outside of my actions within DT. Projects or partial projects that involve frequent structure (folder/group) or file changes such as fresh data files or generated output or code … I leave those out of DT. Unfortunately, this leaves me with two places to search and modify throughout the day (both DT & filesystem) but it’s on a per-project basis so not complex as it might be with a “hybrid” of groups plus indexed folders within a given project in DT.
Greg_Jones:
Indexing documents in DEVONthink 3 is very different than indexing documents in any prior version of DEVONthink. The integration with the Finder is much tighter now, to the point that one could say if you lose indexed files in DEVONthink today, then you would have lost them in the Finder anyway.
Thanks. This is how I thought it would work. If I lose a file in the OS filesystem then I don’t expect DT to protect me from that. But if I move it around in the filesystem anywhere within a large number of folders that are all indexed somewhere in DT then I’d hope it would be tracked/found/indexed by DT soon, if not immediately. Ditto for new additions or renamings anywhere nested within folders that are indexed.
I think the best thing for me to do is index something complex and try some edge cases and see what happens.
I just set up a brand new indexed folder in DT3, did some file additions and deletions which could happen in the real world, and I got the famed File Missing error in DT3:
Where is the file by the way? It is in the Finder Trash. If the user in this situation empties the Finder trash before realizing the indexed file is missing, then the file will be truly gone.
The point is in the finder if you see a list of 100 files, it means the files are all there. If suddenly the list has only 50 but you expect 100, then you know you just did something wrong and can probably fix it right away.
But In a DT3 indexed group, you might see that reassuring list of 100 files day after day but 6 months from now when you need them and try to select them, you might realize that 50 of them have been “missing” for quite some time.
No the point is files do not move themselves to trash. If you accidentally delete files in Finder then you’ll see it there. Whether they are indexed or not doesn’t make a difference - you have to take care when deleting. It’s that simple.
It’s not about “files don’t move themselves to the trash” - it is simply a fact that under some conditions a file can still remain displayed in DEVONthink even though it is in the trash, and that can catch some people out.
There are benefits to either way of working, and no-one is suggesting one is right or wrong.
But for some users internal is inherently a safer option than indexing.
As we always have said, indexing should be carefully considered, especially with the tighter Finder integration. There can be benefits to both approaches but indexing requires more care.
Hi,
I’ve been using DT3 for a couple of months (still lots of things to learn). I work like that, indexing documents that live on Dropbox. Also sync to Dropbox from DTtGo.
My golden rule is NOT to TOUCH the original documents. All the interaction with the ‘raw’ documents (mainly on a PDF format), is done via DTtGo. Although, sometimes, I did open, read and highlight a PDF on Acrobat, for instance, and saved it afterwards. DT3 can read the changes successfully.
So far so good. Maybe this goes against all the DT3 rules… if so I’d appreciate tome feedback.
