Heads up when using Arq to backup Devonthink

mailbuoy · May 11, 2019, 12:04am

First, thanks for a great update! I have switched from DTPO2 to DTP3.

I am using Arq Cloud Backup - just started one-month trial. I noticed that my DT databases in Arq Cloud were only a small fraction of their actual size, so I “restored” one of the databases as a test. The group structure looked fine but there were no documents!

It turns out that Arq | Settings | File Selection | Exclude Files/Folders defaults to exclude folders ending in “.noindex”. The fix was easy. Remove the exclusion from the list and Arq picked up the missing 32 GB!

So, just a heads up.

BLUEFROG · May 11, 2019, 1:59pm

Welcome @mailbuoy

Our company president uses Arq and is likely aware, but I didn’t know this as I don’t use it. Thanks for the heads-up so other users can avoid potential problem too!

JCSully217 · May 11, 2019, 2:16pm

So there is no confusion, if you use Arq’s own backup cloud service there are exceptions added.

If you use another cloud Destination such as Amazon S3, Backblaze B2, Wasabi, you do not need to worry about this, because those require you to add exceptions, there are no exceptions by default.

OttmarKlaas · May 11, 2019, 2:37pm

I can confirm that. I was checking my setup (Arq with BackBlaze) and did not see anything missing from the backup with the default settings.

vinschger · May 11, 2019, 2:42pm

can your CEO (do you mean @cgrunenberg ?) officially confirm, that these exception has to be removed from ARQ in order to have valid backups?

Does DT have to be closed during backup in order to allow a correct backup? Or is it no problem to backup a database with ARQ, while it is opened in DT?

BLUEFROG · May 11, 2019, 3:03pm

No, it’s @eboehnisch and he would have to comment on it.

eboehnisch · May 11, 2019, 4:43pm

I’m using Arq with Amazon S3 which has no default exceptions. I just added some myself for other files and folders that don’t need a backup.

Yes, you’d need to remove a possible exception for “.noindex” or the actual files will not be backed up.

Thank you for making us aware that such an exception even exists. I wasn’t aware of this.

vinschger · May 11, 2019, 4:51pm

thanks. can arq correctly backup a DT database, while it is open in DT? or does it need to be closed in DT fir a correct backup?

Linn · May 11, 2019, 5:18pm

I really appreciate this heads up! I got Arq on Eric’s recommendation for a three-month foreign trip, and I wanted rock-solid backup. And I appreciate the reliability of DTP because I never had to discover this issue with .noindex files.
Thanks!

eboehnisch · May 12, 2019, 11:27am

From my experiences it backs up databases also when DEVONthink is open but, of course, when you close DEVONthink it might update files and so the backup would need an update too.

Here I just let Arq make snapshots every hour and one manually triggered at the end of the day.

evank · May 17, 2019, 1:14pm

Can you point to where to find this in Arq? I haven’t been able to find instructions for locating exceptions on their help page. Thanks.

JCSully217 · May 17, 2019, 1:22pm

If you are using Arq Backup (not cloud), here are the instructions. Not sure about cloud, as I only use BackBlaze B2 and Wasabi.

evank · May 17, 2019, 1:27pm

Thanks. I had found that, but there were no exceptions listed when I clicked “Edit backup selections . . .”

I assumed from the earlier post that “.noindex” was excluded by default. Did I misread that?

JCSully217 · May 17, 2019, 1:29pm

Which service are you using to backup to? Arq Cloud or a Third party service?

mailbuoy · May 17, 2019, 1:44pm

If you are using Art Cloud Backup you will find exclusions under Settings:

Settings | File Selection | Exclude Files/Folders

evank · May 17, 2019, 2:12pm

I’m backing up from my office to my Synology NAS at home. I believe I set it up via webDAV.

JCSully217 · May 17, 2019, 9:32pm

Okay, so you won’t have any exclusions by default, meaning your dbs are backing up correctly. It’s only the Arq Cloud that has the exclusions by default.

evank · May 18, 2019, 2:14am

Awesome. Thanks for clearing that up.

awmn · October 22, 2019, 7:42am

Hi all,

I’ve had Arq Cloud Backup for a week now and checked the exclusions.

Files ending in “.noindex” were not in the exclusions window.

I’m guessing this is now not a thing to worry about, right?

Anyone else in Arq/DT land can confirm?

eboehnisch · October 22, 2019, 8:37am

Folders or files ending in .noindex are marked so that Spotlight doesn’t look inside them. You might still want to back them up, like with the Files.noindex folder inside of DEVONthink database packages.