Enhance external PDF Data Analysis with AI: A DEVONthink Pro Script for Callback URLs

I would like to share a script for a smart rule in DEVONthink Pro. Perhaps someone else has similar requirements and can benefit from it.

Broader Applicability:
This script can be generally useful for any scenario where external data analysis (i.e., not in DEVONthink Pro) involves a large number of PDF files, and it is helpful to reference the original file in DEVONthink Pro from a table of analyzed data.

For example, it could be used for scientific research where PDF files are analyzed externally (text-based analysis only) and the original PDF files need to be referenced later, or for legal case management where documents are reviewed and annotated externally, and the original documents in DEVONthink need to be accessed.

Ideally, I would prefer not to need the workaround described below with Google’s new “NotebookLM”. If AI functions are indeed coming to DEVONthink Pro, I hope for the ability to apply a prompt to selected PDFs and have the analysis table automatically output in DEVONthink’s table function (.tsv). This would likely eliminate the need for external data analysis tools or AI tools in many cases.

My Example Process:
I upload invoices and bank statements to Google’s “NotebookLM,” which analyzes them and, with a detailed prompt, provides me with a detailed table for my personal accounting in seconds, exactly how I need it. Since “NotebookLM” creates closed AI notebooks, it only uses the provided data. If the source data does not yield any results during the analysis, the AI simply outputs no result, and the source data would need to be completed first (if desired) or the prompt must be adjusted accordingly to handle missing data in the results table.

Thus, there is no annoying information noise or hallucinations like with ChatGPT. The provided sources behave like in a small, closed ecosystem. The table generated from the analysis is available in Markdown format, so I can retrieve and process it from “NotebookLM” in seconds.

My Previous Problem:
Since “NotebookLM” converts PDFs to text for analysis, I could not easily access the original invoice file later. It doesn’t help to have the callback URL in the file metadata or Finder comments. As a result, I had to manually find the original invoice file in DEVONthink based on the table data if I wanted to look up something in the original PDF.

Losing time at this stage, even though I could now handle my previously manual process in seconds with NotebookLM, was something I wanted to avoid.

Solution:
I described my problem to an AI and received this AppleScript, which I integrated into a smart rule. This rule has only one condition, “Kind is group,” and is triggered manually when I select one or more groups. The smart rule then starts this AppleScript.

Shortly after, I have a folder on my desktop with the PDFs of all previously selected groups. Each of these helper PDFs for “NotebookLM” has an additional page with the callback URL for the original file in DEVONthink, thanks to the AppleScript. “NotebookLM” then creates an additional column in my table for the link to my original file in DEVONthink.

Since the script is an AI result, it may be optimized further to achieve the same outcome. I haven’t checked it in that regard as it meets my requirements.

For those concerned about data privacy:
“NotebookLM” states that it does not use uploaded data for AI training purposes, only potentially for program optimizations. This makes sense to me, as only data that you upload yourself can be analyzed. The AI does not refer to other sources in its analysis.

But of course, I would feel more secure if I could eventually map this data analysis process entirely within my favorite program, DEVONthink Pro… :wink:

on performSmartRule(theRecords)
	tell application id "DNtp"
		-- Wählen Sie die aktuell ausgewählten Gruppen
		set theSelection to the selection
		if (count of theSelection) is less than 1 then
			display dialog "Bitte wählen Sie mindestens eine Gruppe aus." buttons {"OK"} default button "OK"
			return
		end if
		
		-- ĂśberprĂĽfen Sie, ob die Auswahl Gruppen sind
		repeat with theGroup in theSelection
			if (type of theGroup) is not group then
				display dialog "Bitte wählen Sie nur Gruppen aus." buttons {"OK"} default button "OK"
				return
			end if
		end repeat
		
		-- Fragen Sie nach dem Namen des zu kopierenden Ordners
		set exportGroupName to text returned of (display dialog "Name des zu kopierenden Ordners:" default answer "Unbenannte Gruppe")
		
		-- Fragen Sie nach dem Speicherort im Finder (Standard: Desktop)
		set defaultPath to path to desktop folder as string
		set targetFolder to choose folder with prompt "Wählen Sie den Speicherort für den Ordner:" default location alias defaultPath
		
		-- Erstellen Sie den neuen Ordner im Finder
		set targetPath to (POSIX path of targetFolder) & "/" & exportGroupName
		do shell script "mkdir -p " & quoted form of targetPath
		
		-- Schleife durch alle ausgewählten Gruppen
		repeat with theGroup in theSelection
			-- Holen Sie alle Dateien in der Gruppe
			set theRecords to children of theGroup
			
			repeat with theRecord in theRecords
				if (type of theRecord) is PDF document then
					-- Holen Sie die UUID des PDF-Dokuments
					set theUUID to uuid of theRecord
					set callbackURL to "x-devonthink-item://" & theUUID
					
					-- Erstellen Sie ein neues HTML-Dokument mit der Callback-URL als Link
					set htmlContent to "<html><body><p>Click the following link to open in DEVONthink:</p><p><a href=\"" & callbackURL & "\" style=\"color:blue;text-decoration:underline;\">" & callbackURL & "</a></p><p>If clicking the link does not work, right-click (or control-click) the link and select 'Open Link'.</p></body></html>"
					set newHTMLRecord to create record with {name:"Callback URL", type:html, source:htmlContent} in (parent 1 of theRecord)
					
					-- Konvertieren Sie das HTML-Dokument in ein PDF
					set newPDFRecord to convert record newHTMLRecord to PDF document
					
					-- FĂĽgen Sie das neue PDF am Ende des Original-PDFs an
					set mergedPDF to merge records {theRecord, newPDFRecord}
					
					-- Benennen Sie das neue PDF um
					set originalName to name of theRecord
					set name of mergedPDF to originalName & "_UUID"
					
					-- Exportieren Sie das neue PDF in den Zielordner
					set exportPath to targetPath & "/" & (name of mergedPDF)
					export record mergedPDF to POSIX path of targetPath
					
					-- Löschen Sie das temporäre HTML-Dokument und das konvertierte PDF
					delete record newHTMLRecord
					delete record newPDFRecord
					
					-- Löschen Sie das zusammengeführte PDF in DEVONthink
					delete record mergedPDF
				end if
			end repeat
		end repeat
		
		-- Löschen der DEVONtech_storage Datei (falls vorhanden)
		do shell script "rm -f " & quoted form of (targetPath & "/DEVONtech_storage")
		
		display dialog "Die Gruppe wurde erfolgreich kopiert." buttons {"OK"} default button "OK"
	end tell
end performSmartRule

This script has at least two major issues:

  1. User interaction is required. This is highly discouraged for smart rules as they’re automatically performed in the background.
  2. It uses the current selection instead of the items matched by the smart rule’s conditions

Following up on @cgrunenberg’s comments, there is no need for a smart rule in this case. It should be sufficient to save it as a script and just run the script when you select the desired groups. This also addresses the comments made.

Thank you both for your insightful feedback!

@cgrunenberg, I appreciate you pointing out the issues:

  1. I understand that user interaction in smart rules is discouraged due to their automatic nature. I’ll look into modifying the script to minimize or eliminate the need for user prompts.
  2. Using the current selection instead of the items matched by the smart rule’s conditions is indeed not ideal. I’ll revise the script to better align with the smart rule’s intended functionality.

@BLUEFROG, thank you for the suggestion. Saving the script and running it manually when selecting the desired groups does seem like a simpler and more efficient approach. This way, I can avoid the issues related to smart rules and user interaction.

Initially, I was just looking for a simple way to make it work, which was my primary goal. The AI provided a solution that, while not perfect, did meet my immediate needs. It’s working without mistakes and it helps to achieve my goal. I’ll definitely take some time to delve deeper into this and refine the script based on your recommendations.

Thanks again for your valuable advice!

1 Like

You’re welcome.

So you end up with one folder full of merged PDFs at the end of your script?

  • Then what do you do with them?
  • And what is an example prompt you’d use?

Thank you, @BLUEFROG !

Yes, at the end of the script, I end up with one folder full of merged PDFs. Here’s what I typically do with them:

  1. Upload to NotebookLM: I upload these merged PDFs to Google’s NotebookLM for analysis. The additional page with the callback URL ensures that I can easily reference the original file in DEVONthink Pro from the analysis results.
  2. Data Analysis: NotebookLM processes the PDFs and provides me with a detailed table in Markdown format, which I use for my personal accounting and record-keeping.

An example prompt I’d use in NotebookLM might be: “Analyze these bank statements and invoices, summarize the expenses by category, and provide a monthly breakdown. Include the callback URL for each document in the final table.”

My actual prompt in NotebookLM is much more detailed. It resembles a script prompt (e.g., Python) in its description and thoroughness to ensure an accurate analysis of the data. The wonderful thing is that NotebookLM allows this level of detail to work seamlessly with natural language. I store the detailed prompt in a text file within NotebookLM and upload it as a source alongside the PDFs. In the chat, I simply instruct NotebookLM to apply my “Instructions” source to all other uploaded sources.

A few seconds later, it delivers a table that cleanly lists the analyzed information from the invoices and bank statements (name, origin (for tax/reverse charge), net amount, VAT, gross amount, invoice date, payment date, type of source (invoice, statement, or both), service period, invoice reason, info (e.g., discrepancies between invoice amount and payment amount), and the callback URL to the original invoice in DT).

I then further process the Markdown table in Numbers. The Numbers document, including the link information to the original invoice, goes into my DT database.

But I could also simply copy it into DT’s table function (.tsv). If there is an AI chat function in the future that can be applied to selected files or groups, this process could be entirely managed within DEVONthink. The key would be the strict limitation to the selected files. This is exactly the strength of analyses via NotebookLM: no hallucinations if the AI doesn’t know how to proceed, resulting in usable analysis outcomes. (And of course, this could be a disadvantage for those who see AIs as answer machines whose sources are training data and online data. NotebookLM is not made for that. It’s exclusively about analyzing your own data.)

Thinking through the entire process initially took some effort, especially crafting the detailed prompt. However, the initial effort is worth it for my recurring analysis tasks. Every subsequent data analysis is completed in no time at all.

I appreciate the support and the opportunity to optimize the process with your insights.

Are you processing multiple groups into one group or multiple groups into individual groups? It appears the former.

With this script, I can select either a single group or multiple groups. The modified helper PDFs are all placed into a single folder on my hard drive, making it easy for NotebookLM to pull them from there.

There is a bottleneck: modifying and exporting large quantities of PDFs can take some time. However, that’s a small price to pay considering how much time this entire process used to take me when done manually or even semi-manually. It’s a significant improvement overall.

Are these invoices or smaller items like scanned receipts too?


I don’t use NotebookLM and just now fired it up. Are you saying you have notebooks that have 50 or fewer PDFs in them, per this message…

Yes, it involves both invoices and smaller items like scanned receipts.

Currently, I upload invoices and bank statements to better and more quickly understand the numbers through analysis. Additionally, the analysis helps me reconcile invoices and bank statements. Smaller receipts and vouchers, which are typically paid in cash, can be easily integrated into this process. The prompt in NotebookLM would need to be adjusted accordingly so that the analysis knows how to handle receipts in the table that contain “paid in cash” or “received in cash” etc.

Each notebook in NotebookLM can contain a maximum of 50 sources. Therefore, I make sure to split my uploaded PDFs accordingly when analyzing larger amounts of documents. In most cases, this limit is sufficient for my analyses. If I have more than 50 PDFs, I simply create additional notebooks.

Ultimately, I use NotebookLM merely as a vehicle for analysis. Theoretically, I wouldn’t need to save notebooks. The data comes from DEVONthink, is analyzed in NotebookLM, and then goes back to DEVONthink (as .tsv or .numbers). I only perform the analysis in NotebookLM. I work with the data in DEVONthink or Numbers.

There is a lot of potential here, whether through Google scaling this experimental tool to meet user needs or another developer making it more comprehensive. Currently, NotebookLM works well for analyses, but there is still room for improvement in terms of functionality.

While NotebookLM is used as my example for this process, the modification (with the callback URL) through the script can be fundamentally relevant to all data analysis processes or tools where PDFs are analyzed in a text-based manner AND file metadata in PDFs cannot be accessed. Tools such as Pandas in Python, R (RStudio), Excel/LibreOffice Calc, KNIME, Tableau, Jupyter Notebooks, and Apache Spark all fall into this category (unless the callback URL is integrated into the analysis process through other automation – many roads lead to Rome, as we like to say here…). These tools generally analyze the text-based data directly and do not use metadata or Finder comments for processing.

The original files in DEVONthink remain untouched, ensuring data integrity. By embedding the callback URL directly into the text of the analyzed documents, the analysis table can contain links to the original files in DEVONthink. This approach ensures that the original data is always accessible and referenceable without altering the source files.

After transferring the PDF helper files (with embedded callback URLs) into the respective data analysis tool, these files can be safely deleted from the hard drive.

Isn’t that what any self-respecting banking and accounting app does since decades? Just asking.

1 Like

Hey chrillek,

thanks for your interesting question! You’re right, many banking and accounting apps have been offering good features for years. The difference with AI-powered data analysis tools lies in their flexibility and extended capabilities:

  1. Individualization and Flexibility: While conventional apps often have predefined functions due to their programming, AI-powered tools allow for highly customizable analyses. I can set specific rules and criteria tailored exactly to my needs. What’s achievable in DT through built-in automation options feels like the pinnacle of what’s possible in an app without AI. And I’ve been grateful for that for many years.

  2. Complexity of Analyses: AI-powered tools can perform deeper and more complex matching rules and analyses that go beyond the functions of typical banking apps.

  3. Integration of Various Data Sources: AI-powered data analysis tools often enable the combination and matching of various document types (PDFs, emails, scanned invoices, text, URLs, etc.) in a single analysis, which is often difficult with conventional apps. DEVONthink has essentially been doing nothing else for decades, at least in its DNA. DT with AI-powered data analysis (without the typical ChatGPT hallucination noise) would be even more of a killer app than it already is, as it would build on the strengths for which DT already has a certain unique selling point. In an AI-powered future, it would only be logical to at least offer this possibility.

  4. Automation and Efficiency: Let’s take the previously mentioned invoice receipt and account transaction reconciliation as a practical example. While banking apps like MoneyMoney or iFinance offer quite good functions, they hit limits when it comes to detailed matching and analysis. NotebookLM or other AI-powered analysis tools can go further here:

  • Automatic assignment of invoices to account transactions without manual entries
  • Processing of various document types in a single analysis
  • The AI can recognize and suggest potential matches based on probabilities, even if they mathematically deviate from each other
  • Significant time savings through automation

So it’s not about replacing existing apps, but complementing them with AI-powered analyses and making the entire process more efficient. The invoice and account transaction reconciliation is just one example of the possibilities in the broader context of data analysis.

In the future, more and more apps will integrate AI to enable such specialized analyses. Until then, external AI-powered data analysis tools offer a way to go beyond the limitations of traditional apps and make data analyses more flexible and efficient.

Thus, I currently can’t avoid thinking about how to get data to be analyzed out of DT with as little manual effort as possible and get the analysis result back into DT within a few minutes to continue working with it in DT or other apps.

What do you think in general about this AI-driven development? Have you had similar experiences or do you see other ways to address such challenges in the future?

In other words: NotebookLM, as example, is designed to precisely analyze the provided data and generate responses based on it. It does not think creatively on its own or add information that is not contained in the provided data (as ChatGPT etc). This approach has the advantage that NotebookLM does not fabricate or provide false information, but it also means that it is not capable of thinking beyond the provided data or generating new ideas. It is only useful when there are already existing data to analyze (as we have in DT). This analysis AI is specifically designed not to create new data.

@Tossn Just to flag, the LLM absorbing your data isn’t the only privacy concern. With your process you’ve just uploaded all your bank details to a system that’s vulnerable to manipulation, as well as the usual malicious attacks, hacking, etc. Whilst “not using your data for training” is certainly one question, it isn’t the only one! How is your data being retained on their servers, and how are they securing user details?

I don’t know your location, but here in the UK you would not be covered if your bank accounts are compromised after you’ve shared bank statements with an unofficial third party.

4 Likes

Hi MsLogica,

Thank you for your important concerns! The reason for my script creation was the limitation in DEVONthink that I couldn’t perform certain data analyses directly within the app. This made me aware of the broader value of AI-powered data analysis tools like NotebookLM.

You are absolutely right that data privacy and security are critical aspects, especially with sensitive information like bank details. In my initial example, I was mainly illustrating the capabilities of AI-powered data analysis tools rather than focusing on the actual use for bank data.

The main point of my posts was to demonstrate how AI can be used for data analysis to perform specific and flexible analyses, and how I efficiently move data between DEVONthink and NotebookLM. Of course, I would prefer if such analyses were entirely possible within DEVONthink to minimize additional security risks.

Regarding data analysis, it is particularly important that the AI analyzes only the uploaded data and does not generate new, creative content. This is a clear advantage in avoiding so-called hallucinations that can occur with other AIs like ChatGPT.

Your concerns about data security are absolutely valid and highlight the necessity for such AI-powered analysis tools to adhere to strict security protocols.

Why should DEVONtechnologies take on that risk of an “intelligent” agent that perhaps may “learn” and step
out of bounds”? Why accept they provide “safer” AI software than anybody else?

These questions are rhetorical but I feel worth asking.

Hi rmschne,

Thanks for your comment. I understand your concerns and want to clarify that my intention was not to suggest that DEVONtechnologies should immediately take on such a risk. Rather, I expressed that I would be excited if DEVONthink could securely implement AI-powered data analysis in the future.

DEVONthink’s core strength lies in data and document management, and I see great potential in combining these strengths with secure AI functionalities, especially as the field of data analysis becomes increasingly intertwined with AI technologies. Of course, it is essential that such implementations are secure and follow strict security protocols.

The discussion about AI security is complex and ongoing. I appreciate that NotebookLM does not use uploaded information for training purposes. This does not make it 100% secure, but it is safer than some other popular AIs. This level of security was sufficient for me to use it and to think about how to automate the extraction of data from DEVONthink and efficiently reintegrate the analysis results.

I hope this clarifies my perspective. Thank you for the valuable discussions and insights.

Hello everyone,

I would like to thank you once again for the valuable feedback. It seems that some misunderstandings have arisen regarding the significance and potential of NotebookLM. This may be because I unfortunately used a seemingly unsuitable and prejudiced use case (@MsLogica). That was not smart, but it does not change the fundamental importance and potential of this connection between DEVONthink and NotebookLM or similar, long-established data analysis tools. Therefore, I would like to take this opportunity to clarify some points and explain the larger context.

NotebookLM in the Context of Established Data Analysis Tools

NotebookLM is a new tool from Google that focuses on analyzing and extracting data from uploaded documents. It is important to emphasize that this type of technology is absolutely not new. Established software such as Astera ReportMiner and AlgoDocs have been offering similar (plus much more) features for years and are successfully used by large industrial companies to conduct complex data analyses. What is new about NotebookLM is merely that the app makes it easy for ordinary private users to perform AI-supported data extraction and data analysis for the first time.

Why did I connect the Callback-URL script with NotebookLM (@chrillek)?

  1. Accessibility and Simplicity: NotebookLM was developed to make the power of AI-supported data analysis accessible to a broader audience. It is aimed at private users, students, pupils, data journalists, and others who may not have the financial resources or technical know-how to use professional tools like Astera ReportMiner or AlgoDocs.
  2. Free and User-Friendly: Unlike many professional tools, NotebookLM is free and easy to use. The limits are generally sufficient for private users and are at least higher than in comparable applications. This lowers the entry barrier and enables more people to benefit from advanced data analysis.
  3. Data Security: A frequently expressed criticism was data security. NotebookLM only analyzes the uploaded data and does not use it for training purposes. This minimizes the risk of data misuse and offers a safer alternative to other AI tools that access external data sources or use uploaded data for training purposes.

Further Arguments for NotebookLM

  • No Hallucinations: Unlike other AI tools such as ChatGPT, which can sometimes generate false information, NotebookLM limits itself to analyzing the provided data. This leads to more precise and reliable results.
  • Flexibility and Adaptability: NotebookLM enables specific analyses tailored to the individual needs of the user. This is particularly useful for customized data analyses.
  • Integration into Existing Workflows: The ability to display results in Markdown. Exports to CSV or Excel are allegedly planned for this app, which is still in the experimental phase. As well as other, additional functions. Markdown already facilitates further processing and integration into existing workflows.

Final Thoughts

The reason for creating my script was the shortcoming in DEVONthink that I could not perform certain data analyses directly in the app. This made me aware of the overarching value of AI-supported data analysis tools like NotebookLM. Yes, the script within the rule isn’t perfect, and I’m now using it directly as a script (@BLUEFROG @cgrunenberg).

I hope that these clarifications help to better understand my concern. NotebookLM is an evolution of established technologies that are now being made accessible to a broader audience. And no, I am neither employed by Google nor do I receive any money from them. :wink: I have merely been analyzing data in a professional context for over 30 years and have been using DEVONthink for about 15 years. That’s the only reason I noticed this new connection opportunity.

Skepticism towards new technologies is understandable, but it is important to recognize the benefits and opportunities they can offer. Of course, always while maintaining the highest possible security for the data (@rmschne).

Thanks again for the constructive discussions so far. I look forward to further valuable contributions and exchanges.