I’ve been struggling with databases that don’t update indexed items. I think I know why, but not sure what to do about it.
The use application is in a law firm. I create a new DB for every case. (I have found that this is just easier than having one huge database with all of our cases showing up as “groups” under that because things get unwieldy when cases get large, and then it is difficult to archive them when the case is closed. It is just easier to have separate databases that can be opened, closed and archived without impacting any other cases. It also make small databases easier to manage if they’re not just a small group inside a huge DB. Some of our cases may have only 30 or 40 documents, and some may have hundreds of thousands of docs, photos, etc.)
In Finder the folder structure looks like:
Smith v Jones
Correspondence
Pleadings
Docket
Etc
So then I create a Smith v Jones DB in DT3. But when indexing DB3 requires me to select the Smith v Jones subfolders in Finder, which it then puts into yet another subfolder in DB3, so the DB file structure in DT3 winds up looking like this:
Smith v Jones
Smith v Jones
Correspondence
Pleadings
Docket
Etc
When I update indexed items, I then have to select each of the sub-sub folders which will then update individually. Unfortunately, if subfolders have been added to the main case folder in Finder, that subfolder doesn’t get captured. Presumably because the new subfolder is not part of what has been previously indexed.
I think that if I had a DT3 DB called Cases, and Smith v Jones (and its contents) was a “group” inside of that, then when I go to update the Smith v Jones index it would capture any new folders inside Smith v Jones.
Is there a way to make a DB like Smith v Jones equate to a “group” so that I can avoid the double file organization?
I use DT3 for legal work as well - not as an attorney but as an expert witness.
I love, love, love DT3 and at this point could barely run my practice without it - but I would strongly discourage use of indexed folders in such a situation.
There are a number of “gotchas” you can read about on this forum where files in indexed folders are lost. Sometimes the loss of the file is not obvious until you go to retrieve a given item and find out it is “missing” or “pending.” Imaging that happening during a deposition or trial or on deadline while preparing pleadings.
I do use indexed folders in some situations for purposes of importing items into my database - such as using Zapier to copy email into an indexed Dropbox folder. But in that situation the indexing is only temporary; I then import the contents of the indexed folder into the database. You can even do that importing via a Smart Rule.
Hard drive storage space is cheap. If you are creating one database per case then you will probably never run into RAM limitations on database size. Why take the risk of indexed folders in this situation? Among other things you want to be able to archive each database for months or years and be confident that when you later access that database, all the files are there. Imagine if you need to access an important document on such a case and you find out that since you last accessed the database you upgraded your computer or reconfigured your network or whatever else changed in your setup and now the indexed data is absent or incomplete.
I take the risk with indexed folders because I need to access databases from an iPad which doesn’t have cheap hard drive storage space, my staff doesn’t use DT3 and I need to have access to current file cataloging when I’m not in the office.
My narrow question is about structuring databases to avoid the issue I have described. I am quite happy with the indexing behavior of DT3 otherwise, so I have no intention of changing that particular aspect of how I use the product. I’m just looking to see if there is a better way to use indexing given the folder structure I have described.
Well for starters you can eliminate this double-folder situation by placing your index folder in the root of your database rather than creating a group and placing the index folder there.
So I tried that already. You may not have noticed the part of my question that says:
“So then I create a Smith v Jones DB in DT3. But when indexing DB3 requires me to select the Smith v Jones subfolders in Finder, which it then puts into yet another subfolder in DB3, so the DB file structure in DT3 winds up looking like this…”
I didn’t create the “group.” DT did. And DT will not allow me to put the folders at the DB root level. So the behavior of the app that I’m trying to avoid is something that the app seems to insist on doing. It’s not my choice to have the folders in the group in the DB. That is precisely what I’m trying to avoid.
Apparently databases require groups of folders and won’t allow document folders themselves to sit at the DB root level.
Perhaps what you mean in this situation is what if your database were named “Test Indexed Folder” so you do not want to have a main parent group with the same name. In that case you could simply index both Subfolder 1 and Subfolder 2 in the database root.
Well you’ve identified the objective, but you don’t seem to encounter the same obstacle that I do.
I just tried it again and it did the same thing again. Here’s how this goes.
I create new DB in DT3. Let’s call it Coke v Pepsi. All good. We have an empty and open DB now.
I then click on File>Index Files and Folders. It now takes me to my HDD in Finder.
On my HDD is my case file. For ease in distinguishing the DT3 DB from the HDD file folder, I will call the HDD folder “Coca Cola v Pepsico.”
I select Coca Cola v Pepsico and click “choose.”
DT3 then indexes all of the files and folders BUT the DB now looks like this:
Coke v Pepsi
—Coca Cola v Pepsico
-----Documents
-----Pleadings
-----Etc.
In short, DT3 ITSELF CREATES a “group” called Coca Cola v Pepsico inside the database called Coke v Pepsi. I didn’t want it to do that. I want the DB called Coke v Pepsi to look like this:
Coke v Pepsi
—Documents
—Pleadings
—Etc.
This is why, when I try to update the index files on the DB called Coke v Pepsi it doesn’t update it. I have to update the “group” called Coca Cola v Pepsico that is inside the database.
That repetitive folder structure and the extra steps in updating the index is the problem I’m trying to solve.
I understand that if I just created a new DB called “Cases” and then indexed cases from the HDD added as “groups” to that DB I would not have the issue. BUT, then I would have a huge database of tons of cases that has to be open all the time, even when I just want to work on one small item.
I was interested to see your post and file structure - I’m also a lawyer and have settled on a similar structure to you with a seperate database for each case and indexed files for pleadings, evidence, expert reports etc.
I started with one DB for all my active cases but it got too big, sync was taking a long time to update on the iPad and I was worried about the fallout if the database got corrupted in the middle of a trial.
Can’t you achieve the result you want by just selecting the subfolders (Documents, Pleadings etc) within the “Coke v Pepsi” matter directory and indexing?
So the process would be:
Select Index Files and Folders… (or Opt + Cmd + X);
Navigate to “Coke v Pepsi”;
Select subfolders in “Coke v Pepsi” (or Cmd + A if you’re indexing all of them);
Yes, you’ve understood it. But the problem with that structure is that for some reason, if new folders are added to the HDD folder after the initial indexing, that structure won’t catch the additions unless one uses that wonky sub-“group” structure.
Example: If I add “Answer” to the Docket folder, and the docket folder was part of the initial index, then DT3 will find it. But if someone adds an entirely new folder to the Coca Cola v Pepsico HDD folder, it won’t show up. So say a case comes in and it is pre-litigation. We won’t necessarily create a pleadings or docket folder until lit is initiated. Once that happens, staff will create a “Docket” folder in the main case file. DT3 won’t find that on its own without the wonky group approach.
In sum: An index update will catch changes inside folders that were ORIGINALLY indexed, but it will not find new folders. Also, if there are “orphan” files that are not inside a folder it will not index those unless, again, you adhere to the DB>Group>Folder structure. Some peope in my office have the habit of just putting docs into the main folder with the plan to put it in the right folder “later.” Bad habit, but there it is. Those docs will not show up.
I think the “wish list” answer would be that if a DB could itself be considered a “group” in DT3 lingo, the problem would not exist. The DB would just be an index mirror of what is on the HDD.
The new folder would show up if you went with the plan to index a main parent Folder in the DB root rather than to index individual folders you place in the root.
It sounds as if you are suggesting that it should be possible for an entire database to be indexed rather than for a designated Group (and its subgroups) to be indexed. I suspect that is unlikely to happen.
You could perhaps get around this by creating an overriding “Index” database which contains a list of your cases and an x-devonthink link to each one. Those x-devonthink links could be to the parent Groups rather than to the database roots.
But the “Cases” database would not at all be huge if it contained only x-item links to your cases. It would probably be trivially small to keep open routinely.
Frustrating. I haven’t experienced that problem to date - my subfolders seem to be updating automatically ok without me having to click “Update Indexed Items”.
I’m a barrister, though, so don’t have other lawyers or admin staff saving documents in my system and probably don’t have the same volume of updating going on.
Yes I have not seen issues with the automatic indexing of items – not that I’m doing it for hand once a month just to catch issues.
In most cases I would suspect permission issues with the files/folders, especially if they are copied from another location with more restrictive settings,
I’m not a lawyer but work in the grey area between lawyer and surveyor. I use DT3 (and Scrivener) for my law library. I keep pdfs in Dropbox with a (parent) folder for each topic, for example LAW (general), LAW- Cases, Law - Books. The folders are each indexed to DT3. By using Dropbox as the intermediary, I can then use DT3 on another machine without any of the transfer issues I was encountering when using the DT3 database on one machine as the go-between for the other.
Since I started indexing folders from Dropbox I haven’t encountered the same issue as the op but I did when folders were in Finder.
Whether it is just as ‘safe’ to keep the folders in Dropbox as in Finder, the other day all the pdfs in one folder in Dropbox went missing - I had to restore from Time Machine. That made me think it might not be such a good idea to keep everything in Dropbox.
I have considered using DT3 in the way you describe. Currently I have a DT3 database for all calls, notes, copy/paste. Everything goes into its inbox and I have smart groups for each client, having tagged each record with my abbreviation for the client’s work. As and when I have completed a job, I can print everything to pdf and keep the information in a central folder and sub-folders I have in Documents. (For contact info, and emails I am using Clio.)
“I understand that if I just created a new DB called “Cases” and then indexed cases from the HDD added as “groups” to that DB I would not have the issue. BUT, then I would have a huge database of tons of cases that has to be open all the time, even when I just want to work on one small item.”
In Dropbox, I have a folder “LAW - Cases” which contains >5000 law reports and digests. I have linked that folder to my DT3 Law database. If I were to keep my work also in that DT3 Law database then by using tags I could tag the case law to a smart group for the client so that anything pertaining to that client would be easy to isolate.
I’ve resorted to indexing all my cases files into a main “MY FIRM” database from OneDrive. It is a massive index but has worked thus far. Always trying to find better methods. One database per case was too unwieldy for me.
