Smart Rule Creating Unwanted Duplicates

I’ve indexed a set of folders into which I drop PDFs from my scanner software (Brother iPrint & Scan).

I’ve created a Smart Rule to monitor each subfolder and “move” and “tag” items “on import”.

The rules are working except that each new item is duplicated twice in the destination group.

I’m guessing that somehow the rule is firing twice before the source PDF is cleared from the inbox by the move operation.

I’ll try to past a screenshot of the rule below, but I didn’t have luck pasting a screenshot above, so we’ll see.

TLDR; I’m trying to replicate the Import Folders feature from Evernote Legacy (which has just been killed off). I’m using my scanner software because a) it has a workflow feature the allows for putting different categories of document into different folders b) I can’t get my Brother ADS-2700W scanner to work reliably for direct scanning into DT3.

Add Move Into Database as the first action.

And just as a suggestion: Why using a tag identical to a group? Either one should be sufficient.

  1. You need to specify criteria in your smart rule.
  2. Is the Brother software doing OCR?

Using Move Into Database doesn’t trigger the rule … because it’s indexed not imported?

The tag is just me experimenting. My first day using the software.

  1. isn’t that what “On Import” is?

  2. Brother software has it’s our OCR the works well enough. I’d rather use (or at least try) the DT3 OCR, but I have 2 problems; 1) when I try to import from scanner I usually get a communication error (why? it can clearly see the scanner, and the scanner works fine with apple image capture) 2) when it does work I don’t get an option to combine all pages into a single PDF, I get separate files for front and back of doc.

My worry is that DT3 doesn’t pair well with Brother, which would make me sad.

Given that I can get the scans into DT3 using the Brother software, and that my initial search results using that OCR seem good enough, it may not be an issue.

Also, using the Brother OCR, may save me $100.

This seems to produce the desired results, indirectly.

I tried adding several additional criteria; kind is PDF, Name begins with Gas, Item is indexed. All produced duplicates, the Kind produced 4 duplicates.

Running the filter duplicates rule, does what it says. But why is it necessary? It doesn’t seem like I’d be the only one trying to accomplish this.

Any suggestions as to a better way of doing this? Possibly including resolving my direct scanner issue. BTW, I have a flatbed scanner on the same network and while I haven’t yet scanned with it in DT3, I can reliably open it. The Brother is a sheetfed duplexing scanner, which is necessary for processing a stack of mail.

  1. isn’t that what “On Import” is?

No. That’s the event trigger. You should be specifically defining criteria to match. This is covered in the Automation > Smart Rules section of the built-in Help and manual.


What other smart rules are you running?


Is Utility Bills an indexed parent group?


Is the Brother software up to date and compatible with the version of macOS you’re using?

What criteria would you suggest I use in order to move any file that I drop into the specified folder as soon as it’s added?

I only have 2 smart rules, one for water and one for gas.

Here’s what the indexed folders look like:

image

DEVONthink Dropbox is a folder on my iCloud Drive, and that’s what I indexed.

If I can make this work, I’ll add several more subfolders for other categories of household documents. As well as the PDF files that currently live in Evernote (somewhere in the range of 20 more folders, if that matters).

The Brother software is up to date and works fine with Apple Image Capture on Sonoma 14.4.1.

I’ll look at the Brother site again for any compatibility issues, but it’s most recent and working fine other than with DT3.

BTW, this produces similar results:

image

Also, I tried removing the OCR with similar results. Attempting to have DT3 apply OCR then resulted in a loop that I had to break if I put the OCR rule first, but when placed after the move it created 2 or 3 extra copies in the trash (after filter duplicates).

I am getting the final result I’m looking for, but the process seems a hash.

When the Brother software does the OCR, what are its settings? For example, is it trashing the original document?

And at a minimum, I would suggest Kind is PDF/PS as the criteria. Even if it would technically work as you have it, you should always strive to be specific in what location you target and what you want to match.

It keeps the PDF and adds an OCR layer:

There are other options:

image

But I prefer PDF.

I don’t disagree that being more specific with the criteria may be better, I’m just pointing out that it works as well without it. Kind = PDF resulted in twice as many duplicates (4) the one time I tested it so far. I did find a firmware update and have been able to direct scan on the first attempt after updating, But the multi-page issue is still there for some reason and that’s a deal breaker. Not sure if the reliability is improved yet.

Testing again:

1 copy in Water Bills group, 2 copies in the trash.

Properly using Kind proves something in your process is incorrect. I think you’re misunderstanding what the Brother software is doing.

Disable your smart rule.
Scan a document.
Check the Finder folder. What do you see in there?

That’s exactly were I started this process, no rule. I get exactly what I expect - a single searchable PDF in the Finder folder and a single searchable PDF in the water or gas subfolder in the DT3 inbox.

This is a process (the Bother part) I’ve been using for years with Evernote Import Folders. The main difference being that Evernote never offered the option to index in place. So, I had it import then delete the original from the file system. 99 times out of 100 I never look at that PDF again, but I know I can find it if I ever need to.

That process allowed you to select which notebook the PDF was imported to. That’s the bit the Smart Rule is up to.

I suppose I could index the subfolders separately … but I guess that would still leave the PDF sitting in an inbox.

Maybe I need to try a completely different approach.

Hold the Option key and choose Help > Report bug to start a support ticket.
Thanks!

Thanks. I was wondering how to do that.

One more bit of testing I did that leads me to question the whole strategy, Instead of deleting everything and emptying the trash between tests, I left the files from a few previous tests in place and added additional scans (some new scans of the same document, some copied and pasted from the Finder Trash). I noticed that the rule seems to be firing and creating additional duplicates of files that had already been moved from the inbox to their respective group.

I haven’t figured out what this means, but I’m thinking an approach that Imports rather Indexes may work better. I can live with the PDF being relocated in the file system by DT3.

still reading the help and Take Control ebook.

Why are you scanning into a folder in iCloud Drive ?

And given the way you have things structured, I think you may be overcomplicating a simple process.

This is the basic process, yeah?…
You scan into Water via the Brother software.
DT detects the scan in the indexed group.
DT moves the document into a group in another database.

Folder is in iCloud Drive to allow other sync’d instances of DT3 to access the contents.

What part of the process seems complicated? I’m wide open to suggestions.

Yes, you have the process.

I’ve found a process that seems to work, but does so by moving the PDF into the database rather than leaving it in the original Finder folder. This is still an improvement over the Evernote process, which renders the file completely inaccessible from Finder.

image

Not exactly what I was after, but I can live with it.

Thoughts?

Using iCloud Drive for this purpose is actually unnecessary if you’re syncing DEVONthink on multiple Macs. You’d use iCloud Drive if you were using the documents in Files.app on Apple mobile or sharing them to others (or potentially other apps) directly.

Still pop open a support ticket, if you would.

I opened a ticket yesterday.

Meanwhile I’m looking over alternate methods of getting my documents into DEVONthink going forward.

  1. Import rather than Index.

  2. In Brother iPrint & Scan I can select the DT Inbox as the destination

  3. Brother iPrint & Scan can also send to the DT application, which pops up a Save dialog where I can add metadata

  4. The firmware update may have stabilized the Import from scanner option, but it still really only works for one page documents.

Index and Move is still my preferred option, requiring the least manual steps after scanning. But options 2 and 3 are workable alternatives. It’s not like I’m going to be scanning hundreds of documents a week, or even a hundred a month.

Going through this process, I am pretty sure I’ve learned that DEVONthink is going to be a good solution for me. Undoubtedly better than Evernote was even before they killed off Legacy.

Next step is to figure out the best way to import all the .enex files I’ve exported from Evernote.

Thanks for all the help so far.

Oh, and I have to say one good thing about Evernote. They’ve given me a full refund for my annual subscription. That’ll offset a good bit of the expense of DT.

I responded to your ticket. Let’s discuss things there.

Next step is to figure out the best way to import all the .enex files I’ve exported from Evernote.

Drag and drop individual .ENEX files to DEVONthink’s dock icon.

Thanks for all the help so far.

You’re welcome :slight_smile:

Oh, and I have to say one good thing about Evernote. They’ve given me a full refund for my annual subscription. That’ll offset a good bit of the expense of DT.

That’s good of them.

Small note: Remember that the price of the software is not an expense. (An expense is recurring, like gas and chai lattes!) It is an investment in your education / business / and maybe even your life (finances, genealogy, health records, hobbies or business ideas, etc., etc.) :)D