Differences between indexing vs. importing files

DEVONthink
1

I’m curious about major differences between importing files from Finder folders into DT vs. indexing them.
I know that I benefit from importing files, such as annotated, highlighted pdfs, to which I can then apply ‘Summarize Highlights’ for a new file version just showing these annotations / highlights, which is very convenient

I can access and work with both imported and indexed files in ‘Workspaces’.

Other than that, I don’t know of another benefit from importing, vs indexing.

It would seem convenient to NOT import files if I don’t see a reason, and just index them … Am I missing something?

  • Thx
2 Likes
2

Did you already see the relevant secton starting on page 52 of the user guide (also available through the in-app Help)? (I was unsure, from your question.)

2021-09-21_21-49-20
2021-09-21_21-49-20992×910 204 KB

Stephen

3 Likes
3

As far as I am concerned, indexing has the disadvantage that DT is not always aware of changes and then needs to be manually triggered to become aware of file movements. I also find it more comfortable to back up databases rather than individual files; my databases are encrypted, so I don’t need to worry about any data being backed up (to WORM or a cloud service) unencrypted.
As such, most of my files reside in DT. I Index some files which I regularly work on from different devices, and which reside on iCloud Drive.
DT does not use proprietary file formats for your files; in fact, it doesn’t change them at all when you import them. It offers an export function which will allow you to export all your files to a mirror image folder structure in Finder if you ever want to leave DT behind for good.
So to me importing makes sense; to me DT feels as if that was what it was designed to do. It’s good at indexing, but (inherently) can’t keep up with some changes which happen when it is closed, for example.

4

I appreciate your distinctions, Blanc. That makes sense to me; I’m a little new to indexing files and I wanted a meta-perspective on this characteristic. I think I’ll stick with importing, too. - M

5

Thank you Stephen. I hadn’t seen that U-guide; I refer to the Kissell book on DT3 and it didn’t have the detailed description on indexing distinctions that the guide has. I think I’ll stick with mainly-importing. - M

6

That is certainly what I do.

Stephen

2 Likes
7

I think of indexing as a way to use DT in a mode where both DT and the Finder are (continually) aware of what each does to any given indexed file within the tree Folder structure. Delete a file in DT; it will be deleted in the Finder. Move it to a different location in the Folder structures at the Finder level, it moves to that revised location in DT.

I think of importing as a way to use DT in a mode where DT initially makes an exact copy of what it is handed from the Finder and then goes to its own room to work on what it has.

–
JJW

8

I tend to Index… I “need” to have my files available in the finder for Google Drive to access them. I wonder if I am missing any features that work better (or only) on imported files? But so far its been working well for me.

9

No; the brilliant thing about DT is it’s flexibility - you can use it as best suits your needs. If indexing is working for you, you’re good!

1 Like
10

One thing to bear in mind about indexing - and I rely on it a lot, it’s a great way to pull arbitrary files and directories into DDTG where you can access them in ways otherwise impossible - is that tags are stored as file metadata. Elsewhere in this forum I discuss making a regrettable (but now recovered) mistake, where I turned off tagging on a bunch of files (I’m a heavy tag user) - the downside of using tags on indexed files is that file requesters in macOS take an age to appear, as it puts together a list of tens of thousands of tags I don’t use in the finder. Just a warning for anyone who decides to take the same crazy path I did - however, most of my workflow is built around indexed directories.

1 Like
11

This has not been my experience, DT and finder are not “equal” partners in this way. Yes, deleting a file in Finder will delete it in DT (once it updates, which is usually automatic in my experience when it detects a change). But deleting the file in DT does NOT delete it in Finder, it just deletes the link to that file (which I think is the appropriate behavior actually). And famously, moving the file in Finder breaks the link to it in DT.

I use both indexing and importing a lot. The main disadvantage of indexing is that if you move or rename a file in Finder, DT loses track of it. But within those limitations it can be a great way to go if you want to make the file available to other apps without having to go thru DT first. If I edit the file with some other app, it is automatically updated in DT. What’s more, the link between file and DT persists across devices: I have access to the indexed file on both my laptop and iPad, just like files that have been imported, but within whatever group/structure I have in DT rather than the Finder.

1 Like
12

I am not sure if I am using different settings than you, but my deletions, edits, movement between folders (as long as that folder is also in DT) etc… replicate both ways between finder and DT. Otherwise I wouldn’t be using it :slight_smile:

13

From the User Manual:

DEVONthink stores individually indexed items by its absolute path in the filesystem. If you move an individually indexed file or folder, it will disappear when the database updates or be reported as missing. This is because the item no longer exists in the indexed location, so the absolute path points to nothing. If you move a file or subfolder inside an indexed parent group, the change should be reflected in the database or the Finder. The parent folder still exists in the same location and DEVONthink can adjust for the changed relative paths within it.

Regarding deletion, that may indeed be some setting that I forgot about and can no longer find. But I’m glad it acts the way it does on my system.

14

I’m with ‘Clearscreen’ on this … moving (indexed) files and folders that contain these files in Finder makes them unavailable in DT. It’s a pain to then have to re-set these.
Please remember if you are using special settings and let us know!

  • thanks
15

As I just wrote in another reply: in my experience, moving (indexed) files and folders that contain these files in Finder makes them unavailable in DT. It’s a pain to then have to re-set these.
Please remember if you are using special settings and let us know!
Thanks

16

I do not think I have any special settings on - but both folders do need to be already indexed in DT… AND, I often have to use the “Update Indexed Items” command … But they do travel between folders successfully as far as I can tell. @BLUEFROG can you confirm the expected behavior? Thanks

17

I remain confused and undecided on this issue. I get that importing has advantages over indexing, and I also get that DT’s “default” operational mode is to import rather than index. But in all candor, who cares?

I need to access documents from a variety of places. I have a virtual machine in parallels, which I use for some work resources, but my primary workspace is in macOS.

The biggest concern that I have about importing versus indexing is that it creates duplicates. And those duplicates don’t sync to one another if I change one. I always have to keep saving it to the DT database every time I make a modification.

But if I happen to pull the document up from Finder and work on it in MS word in my Parallels virtual desktop, and then save it back inside Finder, how does DT even know that it has been modified? DT is looking at a different file than the file that I just modified.

I need consistency across my databases and document repositories.

But in all honesty, I feel like I’m missing something obvious.

I don’t mind re-indexing folders. It’s not difficult. It’s not complicated. But trying to keep two completely different document repositories in sync makes my head explode.

My work life is such that I can’t limit document access to DT. Frequently, I have to get documents from other resources, and put them back into those other resources for others to look at. Not everyone uses DT.

For example someone sends me a document in Google Docs, I make real-time changes and those real-time changes automatically sync to everybody else who’s got access to the document. That can’t happen with DT.

What am I missing?

18

But in all candor, who cares?

Everyone building a database should care and approach the concept carefully and thoughtfully. Did you read the In & Out > Importing & Indexing section of the built-in Help and manual? There are solid technical reasons why importing is the default behavior and indexing is not.

But if I happen to pull the document up from Finder and work on it in MS word in my Parallels virtual desktop,

I’m curious why wouldn’t you use Word in macOS?

The biggest concern that I have about importing versus indexing is that it creates duplicates. And those duplicates don’t sync to one another if I change one. I always have to keep saving it to the DT database every time I make a modification.

Of course importing makes a duplicate as it’s copying into the database. The intention isn’t to import a file into DEVONthink then continue using the file in its original location in the Finder. It’s to use the document in the database just as it was used in the Finder.

While we don’t run Parallels in-house (and it’s beyond the scope of our support), it should be able to access the Mac filesystem. That doesn’t necessarily mean you have to dig into the internals of a database either. I would go into the VM and try to open a file on the Mac, then drag and drop a file from your database into the Open dialog and see how it behaves.

For example someone sends me a document in Google Docs, I make real-time changes and those real-time changes automatically sync to everybody else who’s got access to the document. That can’t happen with DT.

You can’t compare Google Docs to DEVONthink. Well, you can but it makes no technical sense. Your Google Docs aren’t stored on your machine. They are stored online in your Google account.

19

Yes. The argument in favor of duplicating a database of stored documents is unconvincing.

Because I am compelled to use a proprietary document repository on a non Mac laptop. So in order to use Word in macOS I have to port the document over to Mac, work on it there, and then port it back over to Windows so that the rest of my team can access it with changes I have made. Indexing allows me to work on the doc where I find it, make my edits, save it where it resides, reindex and all is good. No duplication. And the argument IFO duplication is what again?

I get it. That’s a value that YOU have. Not me. I either do not WANT to work on the document in the DT database, or for reasons that are not relevant here, I can’t do that.

Well, of course I can. And I am doing precisely that.

I’m well aware of where they’re stored, but thanks for being patronizing. And yes, I can compare them. One works with my corporate workflow and one does not. I don’t care if it makes no technical sense or makes no sense to you personally. Your insistence and rigidity make no workflow sense. Your product is a tool; not a goal.

Your job is to work at DT. To promote and develop YOUR company’s products. Mine is to work where I work. To promote and develop MY company’s products. Hate to break it to you, but your software is just a tool. Like a small Allen wrench in a large auto mechanics shop. When the day comes that your company swallows mine, then we can talk. But until then, we’re living in different worlds.

Your implied insistence that IT departments in companies that have different reasons for being in business should comply with your world view which is limited to whatever DT needs, thinks or wants, is just silly. The rest of the world doesn’t really care about your corner of the IT basement.

So to summarize: Your answer to my inquiry is just that my workflow is wrong and that I should tell my corporate IT folks that Bluefrog says they’re all idiots and that they really need to get on board with the DT way of thinking. Got it. If I can remember where that Allen wrench got tossed I’ll let you know how that goes.

In the mean time, if anyone has any HELPFUL suggestions–this being a user forum not a platform for fanboy ranting–I would love to hear them.

20

I’ll leave @BLUEFROG to respond, if they see fit, to much of your post, but I’ll just make a few of my own points based on my experience with DEVONthink so far.

Your system is perfectly valid if it works for you. Don’t like importing? Don’t.

You don’t need to “get it” in the sense of “Yes, I understand how this works and I can see how this will benefit me so I will use it", because I think you already “get it” as in “Yes, I understand how this works and I can see that this will not benefit me so I will not use it.”

In fact, I posit that, generally, “getting” something is "understanding how this works and whether it will work for me or not”.

Don’t forget, at its heart DEVONthink is explicitly a Mac-only application which is not designed for accommodating direct cross-platform compatibility with shared file areas of virtualised environments (a specialised and niche application), and has certain design decisions incorporated which may not be compatible with how you want to work.

Lastly, if we use the “does this statement make sense when reversed” test (yes, I know, this is not always a valid test, but I think it applies here), you may as well say "Parallels should be changing its virtualisation tools to work with DEVONthink”! (Same with the developers of your employer’s bespoke filing system in Windows, BTW.)

So, I don’t think you’re “missing” anything – just keep indexing, it seems to work for you.

Sean

1 Like