Accidentally imported entire Mac hard-drive to DT

DEVONthink
1

Today I have done something terrible. This is entirely user error (that might be generous, let’s call it “user stupidity”) and I’m just sharing what I did so that no-one else does it, as I cannot see this being mentioned in the forum before.

DT was open, and I was clicking about in my Mac library. I was going to look for my Markdown CSS stylesheet for DT, but I was in the Applications folder and mooching about as one is inclined to do. Whilst in there I clicked on DT whilst holding down the Option key (for no reason at all, I was just clicking about). As I said, this is user stupidity. (I had clicked a couple of other things prior to this, but I think it was this action that triggered what happened next.)

The DT icon in the taskbar immediately got a red pending icon on it, and I had that sinking feeling you get when you’ve just done something you know you shouldn’t have done. I opened DT, and discovered that it was attempting to download the entire Mac harddrive, all the software files, everything. The log was of course going crazy because lots of files are in an unsupported format…

Then to top it all off, I have iCloud CloudKit running, so it has pushed all those files to the sync store…

Weirdly, it also applied every tag it could find in my database, including a bunch which weren’t in my database and I assume are native to Mac. My Inbox now has 6000+ items and looks like this…

Whyyyyyyyyyyy
Whyyyyyyyyyyy1920×716 200 KB

I assume a manual delete (which I have been gradually doing) will sort this out, although I’m not sure what to do about my sync store - I assume the deletion of the files on the Mac will fix that? (Otherwise it’s going to be full of redundant files which will try to index to my iPhone and iPad.)

1 Like
2

Hmm actually it has just occurred to me, as DT has imported these files, does that mean they no longer exist elsewhere on my Mac? If I delete them (as I was doing), are they going to be gone forever? Some of the files are pretty integral, e.g. the entire font library is now in DT, all Firefox’s code files seems to have made it across, etc.

I think I will not touch anything further until someone far wiser than me shares their thoughts!

1 Like
3

Well, you could just have a look into your application folder (for example): if all your programs are still there, DT did not move them.

Or maybe have look in the documentation.

4

Importing copies files into a database.

Whilst in there I clicked on DT whilst holding down the Option key (for no reason at all, I was just clicking about)

?
Option-clicking what specifically? Option-clicking the dock icon does nothing but switch to the app.

5

Do you mean the DT documentation? They sadly do not have a section for people who are idiots. The only thing I could find that may be relevant is that moving items within DT whilst holding down the option key creates duplicates. I’m not sure I would expect the same action outside the app to trigger the same behaviour, but that is the best case scenario for me since that’s easily rectified. (I also was not dragging files about.)

I did wonder if this is a behaviour that’s in Monterey somehow, but I couldn’t find any explanation of what is “meant” to happen if you hold the option key down whilst opening an app from the app folder (probably because you’re not meant to do it…). Obviously I’m not going to try it again with another app to see what happens… :joy:

I have checked my font book since I know how to do that and that seems to all look normal. I can’t check all 6000+ files that I imported though so there’s little way to know what havoc I might have caused. Firefox is at this time still working but I don’t know how to check that its root files are in the right place. I have not rebooted my MBP or deleted any files from DT once it occurred to me that I could be making the situation worse.

6

Option-double-clicking an application in the Applications directory has no unexpected effects.

Option-dragging in the Finder and DEVONthink does duplicate files.

1 Like
7

We overlapped then, sorry Jim! This is excellent news! Copies are easily remedied as I can just delete them all!

I “option-clicked” Devonthink actually within the Applications folder whilst in Finder. Not in the dock. I do not recommend that you try it yourself :grimacing: (Specs if you’re interested are Monterey 12.3.1 and DT 3.8.3.) I think this must be what triggered the action as nothing else was selected at the time and the dock icon almost immediately presented a loading icon that turned red (that’s not an icon specific to DT I don’t think, it’s the loading icon that Mac presents when its doing something with an app).

Once I realised what was happening, i force-quit DT (trying to close it didn’t stop it, I guess because it was in the middle of a big task!). That stopped the import at 6000+ items, but reopening DT then synced the files to CloudKit.

8

No worries!

I would disable the database from syncing on any syncing device and clean the sync location so you can sync afresh.

9

I think I’ve lost a step somewhere a long the way as it’s not working like anticipated. Resuming CloudKit sync is making it try to download the files again. This is what I’ve done:

  1. Delete all the erroneous files in the database on Mac
  2. Empty DT trash
    [2b. Empty Mac trash for good measure]
  3. “Untick” CloudKit sync (no sync options now ticked) on Mac
  4. Reboot Mac
    [All is working fine, yey]
  5. Turn off CloudKit sync on iPad DTTG and iPhone DTTG (in meantime I did notice they both tried to download the files)
  6. “Hard close” DTTG on both devices
  7. Re-open DT on Mac, enable CloudKit
  8. Re-open DTTG on iPad and iPhone

This queues up the 6000+ items I’ve already deleted, so I’ve disabled it again.

Which bit have I missed?

10

There’s no step where you cleaned the sync location or the database in the sync location.

Disable the sync in DEVONthink To Go.
In DEVONthink’s Preferences > Sync, select the sync location, Control-click the database in the Databases list on the right, and choose Clean Database.
After the clean has been successfully reported in Window > Log, enable syncing the database again.
After the sync is finished, you can resume syncing in DEVONthink To Go.

11

Thanks for your prompt reply, Jim. It didn’t work though :worried: I assume I am being a numpty (or more of one).

I have:

  1. Disabled sync on iPhone and iPad (there are no other mobile devices syncing)
  2. Disabled sync on Mac
  3. Cleaned all databases and Inboxes. All reported as successful. (My mishap actually only affected one database, but I cleaned all just to be safe.)
  4. Turned sync back on on Mac. It immediately starts trying to re upload the deleted files :thinking: I haven’t got as far as turning sync back on in DTTG.

I re-read your instructions, noticed you didn’t tell me to turn off Sync on the Mac, so I went through the process again with that switched on, it’s made no difference.

I’ve cleaned the database, and run verify and repair. Log is reporting no problems.

I have however noticed the following. The tag count is correct (AFAIK) in the navigation bar on the left. However, if I look at it in the database properties, I think it’s showing the incorrect count which includes all the tagged files created during my mishap.

Strange tags
Strange tags1798×768 125 KB

I have definitely deleted the files and emptied the trash, and the tag list in navigation isn’t showing all the random tags that were created.

What should I try next? I’m really regretting my life choices today.

12

Hold the Option key and choose Help > Report bug to start a support ticket.
Thanks!

13

Oh no, that’s serious! I have done that, thank you.

There’s no rush with this by the way, it’s Sunday and I’m aware this is a mishap of my own making! Thank you for all your help today.

1 Like
14

I’m sorry to hear this has happened to you; good luck fixing it (Jim’s usually good luck enough). Drop a message here to let us all know you are back up and happy, pls :slight_smile:

15

You’re very welcome. :slight_smile:

16

I’m all fixed. It took a bit of work as the sync was not happy about the files I’d tried to import, but @BLUEFROG got everything up and running again. Thank you for all your help.

3 Likes
17

So what was changed, to fix the issue?

18

Hi Sanug, lovely to “meet” you.

This is of course just crisis advice, it would be far better to never import your hard-drive into DT.

I didn’t let this import complete (which I assume would’ve been worse), but DT is very fast so it had managed to import 6000+ files by the time I stopped it (I had to force quit to do this). Because it’s nothing if not efficient, DT also managed to push many of these files to my CloudKit sync store before I managed to disable this. I disabled this before the sync store could push to DTTG (I think, see below for a caveat).

This left me with 2 problems:

  1. My Mac database full of app files
  2. My sync store with the “bad” version of the database

On Mac, following the confirmations in this thread, I deleted these files, and had no adverse problems on my Mac or DT. However the import did also bring in what I assume were native Mac tags (hundreds!). This is where I got lost in this thread and had to put in a support ticket. These were nested in one of my “real” tags and took @BLUEFROG’s detective work to track down and delete.

The sync store was more problematic, but I don’t really know why. I had to erase the sync store and then re-sync with the Mac. Once that was done I turned on sync in DTTG. My iPhone was fine and re-synced promptly, but my iPad kept getting stuck for some reason (saying items still needed to upload to CloudKit). In the end, Jim advised that I disable sync on the iPad and wipe the databases on that device before turning sync back on, and that worked. It’s possible that my iPad had tried to sync some of the erroneous files before I disabled sync, and that’s why it wouldn’t work when I turned sync back on.

I’d like to think I would’ve figured out the nested tags problem eventually, after much swearing and cups of tea, but I wouldn’t have been able to get sync working, and Jim is a star :blush:

I suppose it’s worth noting that DT itself (and I think the native Mac iconology) immediately alerted me to a problem, as I got a red loading symbol on the app in the task bar. I wasn’t in DT at the time but when I opened it to see what was happening the log was full of errors and the import was in progress. I was saved from a bigger catastrophe by whoever designed the error messages :joy:

I’d like to say I learnt my lesson, but I do like clicking on things to see what happens…

1 Like
19

Did you ever consider just restoring from your last time machine backup, or your carbon copy cloner backup?

Presuming that one of those was operational, I think restoring files would obliterate the tags that caused you issues.

I wonder how backup/restore operations affect DTTG.

20

There is a section called

ID. Ten. T.

We are all members as all of us have done the same sort or thing at some point
Now how do I remove all my merged email files ?

:slight_smile: :grinning:

R

ID. 10. T