DEVONthink 4.3.2: OCR'd PDFs stay "PDF document" with 0 words — three interacting text-extraction bugs

DEVONthink Feedback
1

DEVONthink 4.3.2: OCR’d PDFs stay “PDF document” with 0 words — three interacting text-extraction bugs

Summary

After running OCR (ABBYY) in DEVONthink 4.3.2 on macOS 27, the resulting record stays “PDF document” with word count 0 — not searchable, silently. The PDF itself has a perfectly valid invisible text layer: macOS PDFKit, pypdf, Spotlight (mdimport -t) all extract the full text. Re-importing the identical bytes into a fresh database gives 0 words again, while a non-ABBYY control PDF indexes immediately — so this is specific to DT’s text extractor choking on ABBYY’s output.

Environment

  • DEVONthink 4.3.2, macOS 27.0 (Tahoe), Apple Silicon
  • PDFs produced by DT’s own OCR (ABBYY FineReader Engine): JBIG2/JPX image + invisible text layer (Tr 3), subset TrueType fonts (ArialMT etc., no FontFile), custom /Encoding objects with /Differences, ToUnicode CMaps

What I tested (systematic bisection, 50+ controlled variants)

Each variant was rebuilt with pypdf and imported into DT while reading word count immediately:

  1. Not the file, post or index: fresh import of byte-identical file → 0 words. Control PDF → 1169 words instantly.
  2. Not the images: removing the JBIG2/JPX image changes nothing; conversely the working files also contain JPX images.
  3. Not the encoding form: replacing the indirect /Encoding N 0 R (Differences array) with /WinAnsiEncoding fixes SOME files (those with natural-language text) but not others.
  4. Not FontDescriptor/Flags/Widths/FirstChar: byte-level dumps identical between working and failing files.
  5. Not render mode 3, NUL bytes in Tj strings, octal escapes, non-WinAnsi ToUnicode targets, xref/object-stream form — all individually exonerated.

The three bugs (each proven with a minimal reproducer)

Bug 1 — ToUnicode CMap poisoning. Two byte-identical PDFs where the ONLY difference is the ToUnicode targets for glyph codes 00/01 (00AB/003E vs 0041/0042) — same content stream, same fonts, same everything: one indexes 169 words, the other 0. Something in the CMap parser state fails silently and kills extraction for the whole document.

Bug 2 — decimal-number collapse. Two consecutive text blocks containing ONLY decimal numbers ((1,85) Tj then (0,3) Tj) → 0 words for the entire document. Integers (185, 3) are fine; decimals surrounded by words (verdi 1,85) are fine. Looks like a number/date tokenizer that fails on comma-decimals.

Bug 3 — language heuristic rejects fragmentary text. A text layer consisting of component labels (K EfiA L JC 0 1106 I CO MIVr EDO) or number tables → 0 words. Adding ONE real word doesn’t help; adding a full sentence does. DT appears to discard the whole extraction when the text doesn’t look like natural language. This explains why OCR’d datasheets/schematics/tables are hit hardest, and why prose documents (e.g. a Couch-to-5K plan) index fine after the encoding fix.

Also observed along the way: in some paths the extractor reads only the first BT-block / clips long lines (a 20-line synthetic layer yielded exactly one line’s words until each line got its own BT/ET block).

Reproducers

Minimal single-page PDFs with a Helvetica text layer (built via pypdf, happy to share):

  • A.pdf: (ii 80 Mat 20 7 nl F k) Tj → 0 words
  • B.pdf: same + oscillator circuit diagram → 11 words
  • C.pdf: (1,85) Tj block + (0,3) Tj block + words → 0 words
  • D.pdf: same numbers as (185) (3) → words counted fine

Impact

Any ABBYY-OCR’d document with fragmentary content (schematics, datasheets, tables, labels) silently loses full-text search in DT. Re-running OCR doesn’t help (same ABBYY output). No error is logged.

Workaround

We rebuild the text layer: extract text via PDFKit, write a new layer (one BT/ET block per line group, standard Helvetica, decimals normalized, fragment-only lines filtered, document title sentence first). All 8 affected files now index as PDF+Text. Script available on request.

Happy to provide the reproducer PDFs and full test matrix if useful.

2

Thank you for the bug report! macOS 27 is still a beta but an example PDF would be appreciated (both before and after applying OCR).

3

Thanks for looking into this! Example PDFs attached - one prose document and one fragmentary document (schematic labels), each before (image-only) and after applying OCR in DEVONthink:

  • c25k_printable_plan - prose (a running schedule). After OCR: 0 words in DEVONthink (PDFKit/pypdf extract 1741 chars fine).
  • GDO - fragmentary (component labels). After OCR: 0 words as well.

Note: both “after” files index fine in other extractors (macOS PDFKit, pypdf, Spotlight’s mdimport -t), and re-importing them into a fresh DEVONthink database still yields word count 0. Re-running OCR produces the same result.

For context, the “before” files are the image-only originals as fed to OCR (the text layer was never present), and the “after” files are untouched OCR output as written by DEVONthink/ABBYY.

before_ocr_c25k.pdf
after_ocr_c25k.pdf
before_ocr_GDO.pdf
after_ocr_GDO.pdf

4

I just tested this on the latest beta of macOS 27 using DEVONthink 4.4. First, your after_ocr documents contain actually a text layer and DEVONthink recognizes & indexes it and is able to convert the documents to plain text.

Then I OCRed your before_ocr documents and the result was PDF+Text as expected. A screenshot of Settings > OCR might be helpful therefore.

5

Here’s the OCR settings screenshot - looks like the standard ABBYY configuration:

Screen Capture 31 Aug 2026 at 12.56.59
Screen Capture 31 Aug 2026 at 12.56.591546×1026 124 KB

Versions here: DEVONthink 4.4 (build 28 Aug 2026) on macOS 27.0 (26A5421a), ABBYY plugin fre12_arm / DTOCRHelper 1.1.26, OCR languages English + Norwegian (Bokmål + Nynorsk).

Interesting: you can’t reproduce on the same DEVONthink version. To narrow it down, I just re-imported the exact same after_ocr_GDO.pdf from the previous post into a fresh Global Inbox group on this machine: word count 0, also after waiting. So same file, same DEVONthink 4.4, different outcome - which suggests something machine- or config-specific in the indexing path.

Could the language mix matter (English primary + Norwegian secondary), or is there any other setting that affects how the text layer of imported PDFs gets indexed? Happy to run any diagnostic you suggest.

6

OCR settings don’t affect importing or indexing but bugs of the PDFkit framework might and macOS 27 is still a beta. Does it work after rebooting?

7

Rebooted - and it still doesn’t index, but the picture got a lot clearer.

After the reboot I re-OCR’d two documents (the prose c25k plan and the GDO schematic). Now:

  • Text is selectable in the document viewer.
  • Selecting and highlighting text produces annotation companions containing the complete OCR text (500 words for c25k, 30 words for GDO) - so live PDFKit text extraction works fine.
  • Yet the word count of both documents is still 0 in the Inspector, even after waiting and re-indexing.

Also: other PDFs in the same databases (including ones with rebuilt text layers) index normally after the reboot, so indexing works in general on this machine.

That seems to isolate the failure to the indexing/word-count path for OCR-produced PDFs specifically - not the OCR output itself, and not PDFKit generally. Happy to test anything else you’d like on this configuration (DT 4.4 final, macOS 27.0 beta 26A5421a).

8

As macOS 27 is still work in progress, let us know whether the next beta will fix this or not. Thanks in advance!

1 Like
9

I had a similar issue with MacOS 26. I fixed it by turning off a DT setting: Settings->Files->Import->Recognition->Make text in PDF documents searchable. No OCR issues with documents ending up as “PDF Documents” when that is turned off. Check that setting to make sure that it’s not checked.

3 Likes
10

Excellent. I turned it off and ran OCR on the documents again. All became PDF+Text.
That was somewhat easier than to make a script for a rewrite of the text layer :smile:

2 Likes
11

After I fixed that setting I went back and re-OCRed all of the PDFs that had previously failed.

Credit goes to Jim Neumann who helped me debug it after I filed a support issue.

2 Likes