Importing Data into Custom Metadata Fields

Yes, csv selected and visible. numberOfColumns() no longer returns -1, so maybe I had something wrong before. I have also simplified the file to three rows and two columns:


But still not running correctly.

EVENTS TAB:

app = Application("DEVONthink 3")
	app.viewerWindows()
	app.viewerWindows.byId(6481).numberOfRows()
	app.viewerWindows.byId(6481).numberOfColumns()
	app.getCellAt(app.viewerWindows.byId(6481), {row:2, column:1})
	app.getCellAt(app.viewerWindows.byId(6481), {row:2, column:2})
	app.search("name: hicks-s-2004-Explaining Postmodernism.pdf")
	app.getCellAt(app.viewerWindows.byId(6481), {row:3, column:1})
	app.getCellAt(app.viewerWindows.byId(6481), {row:3, column:2})
	app.search("name: ricoeur-p-2024-Lectures on Imagination.pdf")
Result:
undefined

REPLIES TAB:

app = Application("DEVONthink 3")
	app.viewerWindows()
		--> [app.viewerWindows.byId(6481)]
	app.viewerWindows.byId(6481).numberOfRows()
		--> 3
	app.viewerWindows.byId(6481).numberOfColumns()
		--> 2
	app.getCellAt(app.viewerWindows.byId(6481), {row:2, column:1})
		--> "hicks-s-2004-Explaining Postmodernism.pdf"
	app.getCellAt(app.viewerWindows.byId(6481), {row:2, column:2})
		--> "{ | Hicks, 2004 | | |zu:4247823:UF46F3FJ}"
	app.search("name: hicks-s-2004-Explaining Postmodernism.pdf")
		--> []
	app.getCellAt(app.viewerWindows.byId(6481), {row:3, column:1})
		--> "ricoeur-p-2024-Lectures on Imagination.pdf"
	app.getCellAt(app.viewerWindows.byId(6481), {row:3, column:2})
		--> "{ | Ricoeur, 2024 | | | zu:4247823:IWTNI9MV}"
	app.search("name: ricoeur-p-2024-Lectures on Imagination.pdf")
		--> []
Result:
undefined

I note in both cases, the first line of data (brown-d-1999…) is ignored. Is that significant?

The names in your sample were files without extension. Now they have extensions. That could cause the problem. And the names contain spaces, which is a problem with the search I wrote.
Changing

const targetRec = app.search(`name: ${name}`)

to

const targetRec = app.search(`name: "${name}"`)

Should fix that. And using filename instead of name in that line might make it find the record with an extension. But then they might not be found if you use the original format in the CSV, which didn’t contain the extension.

And the first row of the CSV is skipped by the script. That was my error. Change the line
for (let i = 2; i <= rowCount; i++)
to
for (let i = 1; i <= rowCount; i++)
(i.e. 1 instead of 2).

Both csv and DT filenames contain extensions.

I have tried change to “${name}” – no change.

When change this to “${filename}”, error message:

EVENTS TAB:

app = Application("DEVONthink 3")

app.viewerWindows()

app.viewerWindows.byId(6481).numberOfRows()

app.viewerWindows.byId(6481).numberOfColumns()

app.getCellAt(app.viewerWindows.byId(6481), {row:1, column:1})

app.getCellAt(app.viewerWindows.byId(6481), {row:1, column:2})

**Result:**

Error -2700: Script error.

REPLIES TAB:

	app.viewerWindows()
		--> [app.viewerWindows.byId(6481)]
	app.viewerWindows.byId(6481).numberOfRows()
		--> 3
	app.viewerWindows.byId(6481).numberOfColumns()
		--> 2
	app.getCellAt(app.viewerWindows.byId(6481), {row:1, column:1})
		--> "brown-d-1999-Tradition and imagination.pdf"
	app.getCellAt(app.viewerWindows.byId(6481), {row:1, column:2})
		--> "{ | Brown, 1999 | | | zu:4247823:F85JTBC3}"
Result:
Error -2700: Script error.

I have also tried removing all spaces from filenames to see if that made any difference, but no.

If you are viewing the sheet (the content record), you could simply do this in AppleScript…

tell application id "DNtp"
	tell think window 1
		repeat with theRow from 1 to (number of rows)
			my searchRecs(get cell at row theRow column "Filename", get cell at row theRow column "Cite")
			(* We are using a handler here as the think window we are talking to can't "add custom meta data".
			It is processing the rows and columns in the sheet. *)
		end repeat
	end tell
end tell

on searchRecs(recName, recCite)
	tell application id "DNtp"
		set allMatches to (search "filename: " & recName in (root of current database))
		if allMatches is not {} or (count allMatches is 1) then
			add custom meta data recCite for "Cite" to item 1 of allMatches
		end if
	end tell
end searchRecs

Thanks so much – this works!

Can I ask a follow-up question? If I want to add other metadata alongside the scannablecite such as the ‘PPP’ column into a custom field with identifier ‘pages’, what would that make the Script look like? I’m looking to add several other fields, assuming it can all be done in a single Script.

1 Like

You’re welcome :slight_smile:

Yes, it can all be done in a single script.
You would need to extend the my searchRecs() handler.

To make it more clear for yourself, you could assign variables, one for each row/column cell you’re using. Those variables are passed to the handler in the my searchRecs() handler call shown here.

			set recName to (get cell at row theRow column "Filename")
			set recCite to (get cell at row theRow column "Cite")
			-- Add other variable assignments as needed, e.g., set recPPP to (get cell at row theRow column "PPP")
			my searchRecs(recName, recCite) -- Don't forget to include other variables when calling this handler

and you need to modify the first line in the handler…

on searchRecs(recName, recCite) -- If you add other variables above, you must also include them in the handler here.

Hopefully, you’ve read and understood the changes I am showing you.


In case it’s unclear, here it is with a little more error-trapping at the front end…

tell application id "DNtp"
	if not (exists (content record)) or (type of (content record) is not sheet) then return
	tell think window 1
			repeat with theRow from 1 to (number of rows)
				set recName to (get cell at row theRow column "Filename")
				set recCite to (get cell at row theRow column "Cite")
				-- Add other variable assignments as needed, e.g., set recPPP to (get cell at row theRow column "PPP")
				my searchRecs(recName, recCite) -- Don't forget to include other variables when calling this handler
			end repeat
	end tell
end tell

on searchRecs(recName, recCite) -- If you add other variables above, you must also include them in the handler here.
	tell application id "DNtp"
		set allMatches to (search "filename: " & recName in (root of current database))
		if allMatches is not {} or (count allMatches is 1) then
			add custom meta data recCite for "Cite" to item 1 of allMatches
		end if
	end tell
end searchRecs

Thank you, Bluefrog. All working, and I have learned a lot.

One query. I have one former metadata field (‘key’) which I no longer need, but which keeps reappearing when I do an import. I have tried both unchecking it in the custom metadata list, and deleting it from the list; there is no longer a column with that heading in the .csv, and it doesn’t appear in the AppleScript. Is there a way of preventing it from reappearing?

For completeness sake (and because I posted stupid things before), a corrected version of the script

(() => {
  const app = Application("DEVONthink 3");
  const docWindow = app.viewerWindows()[0];
  const rowCount = docWindow.numberOfRows();
  const columnCount = docWindow.numberOfColumns();
  for (let i = 1; i <= rowCount; i++) {
    const name = docWindow.getCellAt({column: columnCount-1, row:i});
	const cite = docWindow.getCellAt({column: columnCount, row:i});
	const targetRec = app.search(`filename: "${name}"`);
	if (targetRec.length > 0) {
	  app.addCustomMetaData(cite, {for: 'cite', to: targetRec[0]});
	}
  }
})()

This version starts out from the first row of the CSV and uses the filename to search for the … well, filename.

Why or? The condition will be true only if count allMatches is 1, but I find it a bit unusual:
If allMatches is {}, the second part of the statement will still be evaluated. So, with or, count allMatches is 1 will always be evaluated. Hence, the first part allMatches is not {} or is not needed at all.

I just verified that: count of {} returns 0. Consequently, there’s no need to check for an empty list in this context, count allMatches is 1 suffices.

So, have solved the ‘key’ issue by changing the metadata type to Decimal Number – i.e. something incompatible with the data in that column, which is alphanumeric.

Another interesting issue is that each time I have done a batch of records (between 3 and 10), one has not transferred the metadata. I cannot see anything wrong with the record. One time, the metadata ended up populating the csv file’s metadata. If you have any suggestions for why this might be happening, I would be glad to hear.

So this seems to be happening each time I run the script. Is there any way of preventing the script acting on a .csv file?

A little more on this. The last record in the .csv file is ALWAYS missed out; sometimes this is attached to the metadata of the .csv file itself, but not always.
Most recently, I did a batch of 13 (11 correct, two missed, both last in the list); another of 7 (6 correct, one missed); another of 20 (13 correct, 7 missed). It’s NOT the case that it’s always the last ones in the list that are missed – with the last import, the 7 missed were scattered through the list.
I haven’t found anything in the records themselves that might correlate to the missed records, but will monitor as I do more of these.
Any ideas?

Providing a few of the CSV documents with missing entries would be helpful.

1 Like

Are you sure that the files mentioned in the csv exist? You could add log statements to the code to see what the search command is returning.

Thanks for your responses. Yes, to confirm, the files mentioned exist – I report the requisite files each time. Here’s one csv:

Cite	Key	Item Type	Publication Year	Author	Title	Publication Title	ISBN	ISSN	DOI	Url	Abstract Note	Date	Date Added	Date Modified	Access Date	Pages	Num Pages	Issue	Volume	Number Of Volumes	Journal Abbreviation	Short Title	Series	Series Number	Series Text	Series Title	Publisher	Place	Language	Rights	Type	Archive	Archive Location	Library Catalog	Call Number	Extra	Notes	File Attachments	Link Attachments	Manual Tags	Automatic Tags	Editor	Series Editor	Translator	Contributor	Attorney Agent	Book Author	Cast Member	Commenter	Composer	Cosponsor	Counsel	Interviewer	Producer	Recipient	Reviewed Author	Scriptwriter	Words By	Guest	Number	Edition	Running Time	Scale	Medium	Artwork Size	Filing Date	Application Number	Assignee	Issuing Authority	Country	Meeting Name	Conference Name	Court	References	Reporter	Legal Status	Priority Numbers	Programming Language	Version	System	Code	Code Number	Section	Session	Committee	History	Legislative Body
																																																																																						,	
{ | Gadamer, 1976 | | |zu:4247823:QAMFH9LK}	QAMFH9LK	book	1976	Gadamer, Hans-Georg	Philosophical hermeneutics		978-0-520-03475-4				translated and edited by David E. Linge.	1976	2021-10-19 16:43:58	2024-04-15 10:51:08								Philosophical hermeneutics					University of California Press	Berkeley and Los Angeles, CA	English	gadamer-hg-1976-Philosophical hermeneutics				Open WorldCat		OCLC: 655365249								Linge, David E.																																											
{ | Gadamer, 1999 | | |zu:4247823:EL5U3FPQ}	EL5U3FPQ	book	1999	Gadamer, Hans-Georg	Hermeneutics, Religion and Ethics		978-0-300-07407-9 978-0-300-17830-2					1999	2024-04-15 10:52:07	2024-04-15 10:52:51			172										Yale Univ. Press	New Haven, CT	eng	gadamer-hg-1999-Hermeneutics, Religion and Ethics				K10plus ISBN										Weinsheimer, Joel																																											
{ | Gadamer, 2016 | | |zu:4247823:IR3VB7TB}	IR3VB7TB	book	2016	Gadamer, Hans-Georg	Hermeneutics between history and philosophy		978-1-4411-5844-4					2016	2024-04-15 10:53:25	2024-04-15 10:53:56			348						The selected writings of Hans-Georg Gadamer / Hans-Georg Gadamer ; edited and translated by Pol Vandevelde and Arun Iyer	volume 1			Bloomsbury Academic	London Oxford New York New Delhi Sydney	eng	gadamer-hg-2016-Hermeneutics between History and Philosophy The Selected Writings of Hans-Georg Gadamer Volume I				K10plus ISBN										Vandevelde, Pol; Iyer, Arun																																											
{ | Lyotard, 1984 | | |zu:4247823:VI3MZJ5B}	VI3MZJ5B	book	1984	Lyotard, Jean-François	The postmodern condition: a report on knowledge		978-0-8166-1166-9 978-0-8166-1173-7					1984	2024-04-15 10:54:31	2024-04-15 10:55:36			110					The postmodern condition	Theory and history of literature	v. 10			University of Minnesota Press	Minneapolis, MN	engfre	lyotard-jf-1984-The Postmodern Condition A Report on Knowledge				Library of Congress ISBN	BD162 .L913 1984						20th century; Civilization, Modern; Knowledge, Theory of; Postmodernism																																														
{ | Mootz, 2011 | | |zu:4247823:MCNTCY9X}	MCNTCY9X	book	2011		Gadamer and Ricoeur : critical horizons for contemporary hermeneutics						Taylor, (George Howard) LOCATION ITEMS UL: South Wing, Floor 4 Available (184:2.c.201.396)	2011	2017-11-11 16:07:24	2024-04-15 10:51:45													Continuum	London		mootz taylor-2011-Gadamer and ricoeur critical horizons for contemporary hermeneutics										Gadamer, Hans-Georg, 1900-2002; Hermeneutics; Ricœur, Paul		Mootz, Francis J; Taylor, George H.																																													
{ | Palmer, 1969 | | |zu:4247823:8XPBF3WP}	8XPBF3WP	book	1969	Palmer, Richard E.	Hermeneutics: interpretation theory in Schleiermacher, Dilthey, Heidegger, and Gadamer		978-0-8101-0027-5					1969	2024-04-15 10:56:02	2024-04-15 10:56:48			283					Hermeneutics	Nortwestern University studies in phenomenology & existential philosophy				Northwestern University Press	Evanston	eng	palmer-r-1969-Hermeneutics - interpretation theory in Schleiermacher, Dilthey, Heidegger, and Gadamer				K10plus ISBN																																																					
{ | Ricoeur, 1977 | | |zu:4247823:WKN7WZHQ}	WKN7WZHQ	book	1977	Ricoeur, Paul	The rule of metaphor: multi-disciplinary studies of the creation of meaning in language							1977	2017-09-03 07:52:08	2024-04-15 10:51:22								The rule of metaphor: multi-disciplinary studies of the creation of meaning in language (R. Czerny, Trans.)					University of Toronto Press	Toronto		ricoeur-p-1977-The Rule of Metaphor The Creation of Meaning in Language														Czerny, R																																											

…for which only Palmer1969 failed and the csv did not attract a cite

Here’s another:

Cite	Key	Item Type	Publication Year	Author	Title	Publication Title	ISBN	ISSN	DOI	Url	Abstract Note	Date	Date Added	Date Modified	Access Date	Pages	Num Pages	Issue	Volume	Number Of Volumes	Journal Abbreviation	Short Title	Series	Series Number	Series Text	Series Title	Publisher	Place	Language	Rights	Type	Archive	Archive Location	Library Catalog	Call Number	Extra	Notes	File Attachments	Link Attachments	Manual Tags	Automatic Tags	Editor	Series Editor	Translator	Contributor	Attorney Agent	Book Author	Cast Member	Commenter	Composer	Cosponsor	Counsel	Interviewer	Producer	Recipient	Reviewed Author	Scriptwriter	Words By	Guest	Number	Edition	Running Time	Scale	Medium	Artwork Size	Filing Date	Application Number	Assignee	Issuing Authority	Country	Meeting Name	Conference Name	Court	References	Reporter	Legal Status	Priority Numbers	Programming Language	Version	System	Code	Code Number	Section	Session	Committee	History	Legislative Body
{ | Taylor, 2007 | | |zu:4247823:ZT5XDWEU}	ZT5XDWEU	book	2007	Taylor, Charles	Modern social imaginaries		978-0-8223-3293-0 978-0-8223-3255-8					2007	2024-04-16 14:02:24	2024-04-16 14:03:57			215						Public planet books				Duke Univ. Press	Durham	eng	taylor-c-2007-Modern social imaginaries				K10plus ISBN																																																					
{ | Arthos, 2009 | | |zu:4247823:YSQJ8EU2}	YSQJ8EU2	book	2009	Arthos, John	The inner word in Gadamer's Hermeneutics						Synopsis: Late in his life, Hans-Georg Gadamer was asked to explain what the universal aspect of hermeneutics consisted in, and he replied, enigmatically, “in the verbum interius.” Gadamer devoted a pivotal section of his magnum opus, Truth and Method, to this Augustinian concept, and subsequently pointed to it as a kind of passkey to his thought. It remains, however, both in its origins and its interpretations, a mysterious concept. From out of its layered history, it remains a provocation to thought, expressing something about the relation of language and understanding that has yet to be fully worked out. The scholastic idea of a word that is fully formed in the mind but not articulated served Augustine as an analogy for the procession of the Trinity, and served Thomas Aquinas as an analogy for the procession between divine ideas and human thought. Gadamer turned the analogy on its head by using the verbum interius to explain the obscure relation between language and human understanding. His learned interpretation of the idea of the inner word through Neoplatonism, Lutheranism, idealism, and historicism may seem nearly as complex as the medieval source texts he consulted and construed in his exegesis, but the profoundity of his insights are unquestioned. In unpacking Gadamer's interpretive feat, John Arthos provides an overview of the philosophy of the logos out of which the verbum interius emerged. He summarizes the development of the verbum in ancient and medieval doctrine, traces its path through German thought, and explains its relevance to modern hermeneutic theory. His work unfolds in two parts, as an expansive intellectual history and as a close analysis and commentary on source texts on the inner word, from Augustine to Gadamer. As such, this book serves as an indispensable guide and reference for hermeneutics and the intellectual traditions out of which it arose, as well as an original theoretical statement in its own right. “Consummately researched, lucidly written, and persuasively argued throughout, The Inner Wordsucceeds brilliantly in bringing to light this neglected but pivotal matter in Gadamer’s work. Arthos is learned in the best ‘humanist’ way, for he succeeds in creating something new of his own that will speak eloquently to all of us.” —Walter Jost, University of Virginia “Gadamer suggests that the Christian idea of incarnation is a key to his hermeneutics, but does not explain his position in a detailed or systematic manner. Arthos brings his considerable knowledge of hermeneutics and rhetoric to bear on Gadamer's insight, recounting the rich intellectual history to which Gadamer gestures, and providing an extended and detailed exegesis of this pivotal point in the third part of Truth and Method. Gadamer's account of 'linguisticality,' Arthos explains, can best be understood through his use of a complex metaphor—the ‘inner word.’ Arthos matches his erudition with clear and clean prose, and his account exemplifies, rather than just describes, Gadamer's hermeneutical philosophy. Any scholar interested in Gadamer's philosophy should have this book on his or her shelf.” —Francis J. Mootz III, William S. Boyd Professor of Law, William S. Boyd School of Law “Arthos's strength lies for me in his careful reading of the sources. He effectively commands the literature on the subject. This work shows in a sophisticated way the legacy of trinitarian theology for philosophical hermeneutics. The very complex task of illuminating the phenomenon of the verbum interius and indicating its centrality for philosophical hermeneutics is accomplished by John Arthos with great sensitivity to the subject matter.” —Andrzej Wiercinski, The International Institute for Hermeneutics About the Author: John Arthos is associate professor of communications at Denison University.	2009	2017-11-11 15:59:45	2024-04-16 13:48:50													University of Notre Dame Press	Notre Dame, IN		arthos-j-2009-The inner word in Gadamer's Hermeneutics										De differentia verbi divini et humani; De natura verbi intellectus; Gadamer, Hans-Georg, 1900-2002. Wahrheit und Methode; Hermeneutics -- Religious aspects -- Catholic Church; Word (Theology)																																															
{ | Taylor, 1989 | | |zu:4247823:QY4LV3FX}	QY4LV3FX	book	1989	Taylor, Charles	Sources of the self : the making of the modern identity							1989	2017-09-08 15:39:47	2024-04-16 13:52:44													Harvard University Press	Cambridge, MA		taylor-c-1989-Sources of the self										Civilization, Modern; Ethics; Philosophical anthropology; Self (Philosophy)																																															
{ | Allen, 2015 | | |zu:4247823:KH8E8Z6R}	KH8E8Z6R	book	2015	Allen, David	Getting things done: the art of stress-free productivity							2015	2024-04-16 13:56:38	2024-04-16 13:58:17													Penguin	London		allen-d-2015-Getting things done																															Revised																										
{ | Arthos, 2014 | | |zu:4247823:H7IS856R}	H7IS856R	book	2014	Arthos, John	Gadamer's poetics: a critique of modern aesthetics		978-1-4725-9157-9					2014	2024-04-16 13:55:02	2024-04-16 13:55:59								Gadamer's poetics	Bloomsbury studies in continental philosophy				Bloomsbury	London	eng	arthos-j-2014-Gadamer's poetics - a critique of modern aesthetics				BnF ISBN																																																					
{ | Gutmann, 1994 | | |zu:4247823:DNC3QVBE}	DNC3QVBE	book	1994		Multiculturalism: examining the politics of recognition		978-0-691-03779-0					1994	2024-04-16 14:04:00	2024-04-16 14:05:14			175					Multiculturalism					Princeton University Press	Princeton, N.J		gutmann-a-1994-Multiculturalism - examining the politics of recognition				Library of Congress ISBN	E184.A1 M84 1994						Minorities; Multiculturalism; Political activity; Political culture; United States	Gutmann, Amy																																													
{ | Taylor, 1992 | | |zu:4247823:CQNNBXAH}	CQNNBXAH	book	1992	Taylor, Charles	The ethics of authenticity		978-0-674-26863-0				"Originally published in Canada in 1991 under the title The malaise of modernity"--T.p. verso	1992	2019-12-23 09:30:40	2024-04-16 13:53:21			142										Harvard University Press	Cambridge, Mass		taylor-c-1992-The ethics of authenticity				taylor-c-1992-Ethics of Authenticity | Malaise modernity.pdf	BF637.S4 T39 1992						Civilization, Modern; Psychological aspects; Self-actualization (Psychology); Social aspects																																														
{ | Taylor, 2015 | | |zu:4247823:BWS8A9FD}	BWS8A9FD	book	2015	Taylor, Charles	Hegel and modern society		978-1-107-11367-1 978-1-107-53426-1					2015	2024-04-16 13:58:50	2024-04-16 14:00:13			176						Cambridge philosophy classics				Cambridge University Press	Cambridge	eng	taylor-c-2015-Hegel and modern society				K10plus ISBN																																																					
{ | Arthos, 2019 | | |zu:4247823:AAXKSCIJ}	AAXKSCIJ	book	2019	Arthos, John	Hermeneutics after Ricoeur		978-1-350-08086-7					2019	2024-04-16 13:59:58	2024-04-16 14:00:31									Bloomsbury studies in continental philosophy				Bloomsbury academic	London New York (N.Y.)	eng	arthos-j-2019-Hermeneutics after Ricoeur				BnF ISBN	121.686																																																				
{ | Dreyfus, 2015 | | |zu:4247823:7Q2PUL9Q}	7Q2PUL9Q	book	2015	Dreyfus, Hubert Lederer; Taylor, Charles	Retrieving realism		978-0-674-96751-9					2015	2024-04-16 14:05:16	2024-04-16 14:08:32													Harvard University Press	Cambridge (Mass)	eng	dreyfus taylor-2015-Retrieving realism				BnF ISBN																																																					
																																																																																						,	

…of which the following failed (and the csv took the reference from allen-2015):
- allen-2015 (first)
- arthos-2014
- arthos-2019
- gutmann
- taylor-1989

I’m aware the csv files have a lot of fields – this is the raw Zotero output.
I can post further examples if that helps – I don’t want to overload the thread.

I suspect you either don’t have the matching files in the database or you have more than one. Either condition is considered an error according to the script.

Do a search for one of the missing item’s name in quotes.

In the script, you could change this

		if allMatches is not {} or (count allMatches is 1) then
			add custom meta data recCite for "Cite" to item 1 of allMatches
		end if

to

		if allMatches is not {} or (count allMatches is 1) then
			add custom meta data recCite for "Cite" to item 1 of allMatches
        else 
            log message "found " & (count allMatches as string) & " for name '" & recName & '"'
		end if

Then have a look at DT’s protocol.

Aside: What you just posted is not CSV but something else. In CSV, commas separate values and strings containing commas must be delimited by double quotes. That’s not the case here.

Over here, I pasted the content into a blank Numbers document then exported .tsv from there :slight_smile:

I pasted it into a new BBEdit document and set its type to “Tab-separated values”. That didn’t do much for me. Therefore, I supposed that the thing was kinda borken. But then – Numbers seems to like it. But there’s no field Filename in that data (unless I’m again not seeing the obvious).

How can the script even work sometimes with this data – it’s looking for a column named “Filename”, AFAICT.

Filename was just in my example document, just like my custom metadata name Cite was my own :slight_smile: