Comparing Apple Vision and OCR

When creating searchable documents from images or PDF documents in DEVONthink Pro or Server, there are two mechanisms that process them: Apple’s Vision framework or traditional OCR. While seemingly similar in output, there are important differences between them.

Vision

Apple’s Vision framework examines an image and recognizes text in it. It stores this in an index, letting you find the document by its text while not affecting the actual file. This can apply to a standalone image or to a page image within a PDF document. However, Vision stores the text locally, so sharing the document with someone, e.g., on a PC with no Live Text function, gives them just the PDF without any text data.

In DEVONthink’s Settings > Files > Import > Recognize, the Make text in PDF documents searchable feature uses Vision to process image-based PDFs (like scanned pages) when you import such documents. The text will be stored in the index of DEVONthink, so if you export the PDF, it will no longer be searchable. Also, you can choose a Vision model in the AI > Transcription > Images settings that processes images, e.g., on import, or via Data > Recognition > Transcribe Text & Notes.

We recommend only enabling these options if you need them, especially as they will process every incoming image or PDF document. You can always use the features on a case-by-case basis.

Traditional OCR

Traditional OCR generates a text layer and applies it to the document, or, more commonly, creates a new document. The processed document can be shared or moved to other devices and remains searchable since the text layer is part of it. This is much more robust and makes it a more broadly useful feature.

Traditional OCR doesn’t happen on import but can be accomplished manually via Data > OCR or by using smart rules. It uses the OCR settings for resolution, output, etc.

How to find out what needs OCR?

In both cases, OCR and Vision, a PDF document may show it has words in it, including in the Tools > Inspectors > Concordance. For Vision-processed files, this information comes from the index, not the document itself. So when trying to identify PDF documents that need OCR, the correct criteria for a DEVONthink search is Kind is PDF/PS and Word Count is 0 since Vision documents have no text layer, therefore no words.

You may also want to have a look at our tip How to Deal With PDF Searchability.

Does that refer to a Spotlight index or something else?

No, our own in this case.

How do ocr and apple vision compare in terms of recognized text quality?

In my experience, OCR is better. Vision has sometimes trouble putting text on one line into the right order.
And afaik, vision supports fewer languages than DT’s OCR.

2 Likes

Make sure you read and understand the entire blog post. Vision is not a broad replacement for traditional OCR. Unless you remain in a walled garden of your own documents, traditional OCR is still recommended.

2 Likes