Setup Question: Why are my files? Mirroring and Backing up files and folders

[Amontillado]

@nottwentyseven

Anyways, when changing a file in either devonthink or just my finder, the file doesn’t properly get update in the other one. But I would like to have everything setup the same at all times.
So if I add new documents to my study folder (via finder for example), I should update and add this file to the dt database. Some goes if I delete a file. Devonthink still has it in the database. Some goes if I change text in a document. I want the updated version in dethonthink. And this doesn’t work properly with my current setup.

I think I can help. In this situation, just work with your file in Devonthink like you would work with it in the Finder. I would delete the copy outside of Devonthink, but do some experimenting to see how that all works.

Unless you have reason not to, import everything you work with. Don’t import your Documents folder or anything another app manages. Like Dropbox, for example.

As others have said, go slowly. It doesn’t take more than an afternoon to get a black belt in Devonthink. Then, of course, you’ll keep finding new features and new ways to work with Devonthink.

Also remember that if Devonthink is doing something that seems complex, it generally means there’s a better way to look at what DT is doing.

In the case of a file you drag into Devonthink (importing, not indexing it), you either copied it to Devonthink (simple drag and drop) or moved it (Command-drag and drop). If you moved it, there’s one copy, the one in Devonthink. If you copied it, you have two independent copies of the file that can be independently edited, one you see in Devonthink, the other in your Finder.

In the case of imported files, Devonthink finds a nice spot for that file inside the Devonthink database - but it’s really simpler than that.

A Devonthink database is a “package”. That’s a regular folder on your disk, but the Finder will show it as one monolithic object. You can see what’s inside by using the Finders Show Package Contents feature, but there’s no need to do that. Render unto Devonthink what is Devonthink’s and let it have sole control over what’s in the Package. Just rest assured your files aren’t consumed by some Borg-like database. They remain viable files in their original form. Storing a file in Devonthink is as safe as storing it with your Finder.

Devonthink stores files in folders inside the package according to its own rules. The groups you see in Devonthink amount to a curated inventory. When you move files from one group to another in Devonthink, they don’t move on disk, they are just displayed in the new group.

Replicating a file flags it to appear in more than one place in Devonthink’s groups. Edit either appearance to modify the single file represented.

Duplicating actually creates another copy of the file.

There are many more features, of course, but just keep in mind that Devonthink is consistent in how it does things and keep exploring.

[chrillek]

@nottwentyseven

so everything is backed up and in reach from multiple devices.

I begin feeling like a parrot, but: Cloud is not backup. Cloud is mirroring, syncing. A backup permits you to get back to a former state – before you changed or deleted a file. Cloud does not. It’s just dumb file storage. Nothing else.

And if I may ask: Why do you want to use DT? What it is, that you can’t do now that you can do then?

In any case, you have two avenues:

• Index your cloud stuff into DT and be aware that you can encounter interesting issues
• Import your cloud stuff into DT and delete (or move to a hard disk) the originals. Only work in DT from now on.

And set up a backup plan. Please. Regardless of your approach. You can’t live without one. Unless you don’t need your data, that is.

[nottwentyseven]

First of all, THANK YOU! Finally someone who doesn’t complain about my choice of handling my data, but actually helps me develop my system for my personal needs. I don’t get why people get pissed over someone doing things different than they would. (??)

Anyways…

@Amontillado

I think I can help. In this situation, just work with your file in Devonthink like you would work with it in the Finder. I would delete the copy outside of Devonthink, but do some experimenting to see how that all works.

I could see myself mainly working from devonthink instead of using finder for my organization and file handling. I just see the flaw of still not being able to access my data from other (maybe non-apple devices). Explanation: As I wrote I’m mainly a student, which requires me to access my stuff from all kinds of devices, my university’s library computers or my phone for example. Using iCloud or Dropbox I can do that, even on a windows or linux maschine. That’s the main reason why I actually need the same data in devonthink and also on some accessible location. Also if my Mac breaks for some reason, I can’t really do anything with the left over data bases using any other device. So it’s just for my security and flexibility. No matter if my Mac breaks, no matter if I aint got no network connection for a month… I need to be able to access my stuff somehow. (and yes, I keep my doc folder from iCloud always synced locally. It’s never in the cloud only, there’s always a local access.) That’s the flaw here, when only using my data inside of devonthink on my Mac.
I just don’t trust anything that’s just stored on my computer or drives. And I need an alternative format for my data in order to use it on windows or linux, just in case my Mac breaks for some reason and I have to wait for a new one in order to access my stuff. So a system that relies on my physical Mac only is not an option. I want to use a cloud service for my data. Otherwise I could do a backup on an external drive, every time I add a new file to my folders, just to keep it backed up.

@Amontillado

I would delete the copy outside of Devonthink

That’s the part that doesn’t work for me, incase I’m in a “I got no Mac right now” situation. And this also limits me on not being able to access my data on pretty much any other device.

I was seeing reddit thread (yes reddit, sorry), where someone was talking about setting up a smart rule to watch a folder for new files being added. Then these would go into a certain inbox in devonlink, waiting there to be sorted, indexed or whatever. I just need help setting up something like this.
In best case also with an option to detect an update text file for example. (I changed something in the text and want to safe the new version.) Or if a file has been deleted in the original folder, then it can be removed from the database aswell.

I just want to know, if something like this is possible. If so, please give a hint on how to set it up.
Thank you

[troejgaard]

@nottwentyseven

For everyone hating on my love for using icloud…

No one is hating on iCloud. I am a happy user of iCloud myself
The problem is that you call it a backup. It simply is not – and it is important that you understand this. As chrillek says, it is for sync.

Maybe you think of it as a backup, because it is “in the cloud”, not on your local hard drive… So if something happens to your hard drive, you might still have access. It can offer some limited assurance. But if storing something (only) in the cloud gives you peace of mind, this is a false peace.

[nottwentyseven]

@chrillek

And if I may ask: Why do you want to use DT? What it is, that you can’t do now that you can do then?

Let me quote an earlier message:

@nottwentyseven

Priority is keeping my stuff safe, so everything is backed up and in reach from multiple devices.
Devonthink is my tool of choice for keep everything organized, finding stuff and also ocr.

Of course there are other tools to ocr my files, using tags and having advanced search, but devonthink seems to be my all in one solution for this.
To answer your question: I mainly want to use it to organize my studies and for organization of learning. I really enjoy the search functions in devonthink, converting files, ocr, setting up rules to organize my stuff faster. Also clipping web articles for my studies, archiving emails, hopefully being able to access all these files from iCloud as well for flexibility.
This is my (so far conceptional) use case for devonthink.

[nottwentyseven]

@troejgaard

The problem is that you call it a backup.

I get that now, sorry for messing things up, but in this case I’m not into it enough to know all the vocabulary. Anyways, I’ve never had any bad experience accessing and storing my stuff using cloud mainly. Again, my data is important for me to have and keep, but it’s no sensitive data. So even if it gets leak… it’s probably just some studies since documents. This won’t break my heart.
So in this case data security is secondary to me. I just need my stuff to be stored somewhere and accessible from multiple devices in common formats, so preferebly no Mac-only database format.

[chrillek]

The only reliable way to have your data in DT and in whatever cloud service in sync is to index said data in DT.
That will give you something akin to links in a Unix file system. You change in one place, it gets propagated to the other. Since there in fact is only one place.

And nobody got pissed. You asked for advice, which was given. You didn’t like that advice, which is fine.
If you’d provided the information about being able to access the same stuff from different locations upfront, the advice might have been more to your liking. Output depends on input.

[troejgaard]

@nottwentyseven

Finally someone who doesn’t complain about my choice of handling my data

I am sad that your experience of this thread is just people complaining at you. But this is what I tried to communicate earlier. You are confused on the terminology – and you come at this with some underlying assumptions taken for granted. From these assumptions certain things look obvious to you. But if people don’t share the same assumptions, it is not obvious to them what you mean.

I only see you and other people talking totally past each other. Muddy communication.

[Amontillado]

Thank you for your kind words, @nottwentyseven .

I’ll have to think about how to support your file updates. I think someone else already suggested indexing. That’s probably the best way. One copy of each file, accessible either via Devonthink or outside of it.

Files that are indexed are referenced from Devonthink, stored in regular folders.

One way to back up Devonthink databases is to export them as web sites. That will give you a tree of directories you can navigate with either a file explorer or a web browser. Or, you can export to files and folders. Note that restoring from web site output is probably a fairly manual process. Restoring from files and folders probably isn’t so bad, although you would lose tags and DT metadata.

With either of those exports you have to expand and select every group. The export functions only apply to selected groups and items. I think replicants are exported as duplicate copies, too.

The reason you shouldn’t think of sync as backup is because if a file is accidentally deleted, it will get deleted from the sync as well. That’s not a Devonthink issue, it’s the nature of synced storage.

Good luck with your studies! I envy you. I’d drop work and head back to school in a heartbeat if I could.

[DTLow]

@Amontillado

One way to back up Devonthink databases is to export them …

Each database is contained in a .dtBase2 file
You shoud be backing up this file, and using it for restores

[Amontillado]

That will probably work, although the .dtBase2 object in Finder isn’t a file. It’s a subdirectory - “package”, actually.

I like to use the export as archive for backups. I also periodically export to web site.

[rmschne]

See the section in “DEVONthink Manual”, “A word about backups”, page 19 of 3.9.8 version. Recommended reading for all DEVONthink users, including this thread.

[BLUEFROG]

Welcome @sixxam

Have you read the In & Out > Importing & Indexing section of the built-in Help or manual?

[chrillek]

Good question. They’ve been told to do that several times here already. Apparently to no avail.

[nottwentyseven]

Thank you!
Is that a way in setting up a rule or interval for indexing on newly added files?
How does indexing behave if you index a set of files where some have already been indexed and some files might got update with changes in text/content for example? Does it create “duplicates” (so 2x the same file or maybe one old and one new version) or does it just overwrite the previous indexing? So in this case the files are not always said to keep their state. Of course I work on my documents and they keep updating, so the index should update as well

[chrillek]

Read. The. Manual.

There’s no point in expecting others to repeat what has been written many times before. And you’ve been told to read the manual several times, so please just do it.

[nottwentyseven]

I got this now, sorry. I got my setup working now. I read the manual about indexing and got it working.

I just indexed the folder of all my study notes and files. everything shows up in dt now.
If I add/change a from the folder using finder, it automatically applies the change in dt.
BUT: If I delete a folder from the actual folder the fie still shows up in dt. Is there a way to set up a rule or anything to remove a file from the database, when it was deleted from its location? When I click on it, it tells that the file wasn’t found, that’s write… but can dt also remove it then?

Q:
How can I set up dt so it tells me about indexed files I deleted locally or even automatically remove them if? The sync works when adding new files to the locations or changing content in existing files.

[BLUEFROG]

@nottwentyseven

BUT: If I delete a folder from the actual folder the fie still shows up in dt. Is there a way to set up a rule or anything to remove a file from the database, when it was deleted from its location? When I click on it, it tells that the file wasn’t found, that’s write… but can dt also remove it then?
Q:
How can I set up dt so it tells me about indexed files I deleted locally or even automatically remove them if? The sync works when adding new files to the locations or changing content in existing files.

See the Indexing and the filesystem > Deleting Files. Also, cloud services don’t always notify other applications about changes to the filesystem hence the need to manually update. And select the indexed parent group in the database and choose File > Update Indexed Items. You can also handle the deletions in DEVONthink, remembering you need to empty the database’s Trash too.

[laurencec]

Same problem, OP, and I’m going to read these replies as well, and I hope my comment might be useful, too:

The idea of having a purpose, and not just throwing all your stuff into DT, now makes sense to me after, yes, throwing all my stuff into DT. And that is what I realized: I am an info hoarder and autodidact so someday I might want to write or teach about some portion of the associations my synthetic thinking might possibly coalesce around.

So, thanks to the poster who mentioned the bit about having a purpose! That gives me some solace as to why I’m still hoarding with this app and trying to learn “why” (rather than how?) to retrieve any of it (wish there were a visual network diagram, maybe like Obsidian) at this forum! LOL

I lost a bunch of DT files after a hardware failure on a Mac - I didn’t try hard but couldn’t figure out how to recover my DT databases. The databases seemed okay but I guess the actual files were needed and the pointers pointed to oblivion. Sort of wish the app developers would have warned about this kind of mistake and make the proper way to set up and back up more clear since computer failures are not uncommon. But whatever…I own my error and…still don’t know and thus am still taking a chance! LOL - maybe I have time to do this…eventually…now that I have a purpose besides hoarding!

Good luck!

[rmschne]

@laurencec

Sort of wish the app developers would have warned about this kind of mistake and make the proper way to set up and back up more clear since computer failures are not uncommon

Sorry for your loss, but the developers do provide backup guidance in the “DEVONthink Manual”, and basic computer backup regime is required and essential, as you notice.

With backups, recovery would have been simple and easy.

If not already start with getting a TimeMachine backup setup and running. Then look at the 3-2-1 Backup regime written about all over the internet.