Custom Fields

Thank you for your wise words, Greg, especially in light of my own comments:

If I wanted to add an article to a database but wanted some way of recording the original source, so that I can later cite the original source when referencing the database article, how would I go about that?

Use the Aliases column? Spotlight comments?

Clearly, I really would like to make use of Devonthink, but just cannot see how some (simple) database usage requirements are catered for with the program.

Please tell me if there is a way to record the original source. Without tags or groups, as my articles come from a million different places and it would be a nightmare to have my view littered with all of these. Also, tags and groups for only one article (as would most often be the case) would not be suitable.

Is this possible?

And please, Greg, no jokes this time. First, because your last one wasn’t funny, and secondly, because it would be best to keep this discussion constructive.

Jon

I was also thinking that, for publication year, I could do the following:

As far as I am aware, there are four b[/b] date fields used by Devonthink:

  1. Date Created
  2. Date Added
  3. Date Modified
  4. Date Opened

A nice little workaround for me would have been to use Date Created as the original publication date (effectively, when the article was created) and keep the Date Added field for when the article was added to the database.

This way I could easily sort by Date Created (Publication Year), regardless of which tag/group I was using.

Is there a simple way to:

  1. Change the Date Created field? As far as I can tell, there is only the option to change the Date Modified field using a script, and then use a second script to change Date Created to the same value as Date Modified. Seems awkward and tedious for multiple files.

  2. Have only the year for Date Created? I suppose I could have all my articles with 1/1/YEAR, but if possible if would be nice to have only the year.

This version of the inbuilt “Change Date” script will change only the creation date. (The “Change Date” script changes both creation date and modification date to the same date.) Install this in your script menu.

-- Set creation date (but not modification date)
-- Created by Christian Grunenberg on Sun Mar 12 2006.
-- Copyright (c) 2006. All rights reserved.
-- Modified by korm 20130202

try
	tell application "DEVONthink Pro"
		set this_selection to the selection
		if this_selection is {} then error "Please select some contents."
		
		set theDate to (date of item 1 of this_selection)
		set theMonth to ((month in theDate) as integer) as string
		set theDay to the day in theDate as string
		set theYear to the year in theDate as string
		
		if (theDate as string) contains "." then
			set defaultDate to theDay & "." & theMonth & "." & theYear as string
			display dialog "Enter creation date in DD.MM.YYYY format:" default answer defaultDate buttons {"Cancel", "OK"} default button 2
		else
			set defaultDate to theMonth & "/" & theDay & "/" & theYear as string
			display dialog "Enter creation date in MM/DD/YYYY format:" default answer defaultDate buttons {"Cancel", "OK"} default button 2
		end if
	end tell
	
	set theDate to date (the text returned of the result)
	
	tell application "DEVONthink Pro"
		repeat with this_record in this_selection
			set creation date of this_record to theDate
		end repeat
	end tell
on error error_message number error_number
	if the error_number is not -128 then
		try
			display alert "DEVONthink Pro" message error_message as warning
		on error number error_number
			if error_number is -1708 then display dialog error_message buttons {"OK"} default button 1
		end try
	end if
end try

Not possible. Dates are yyyy-mm-dd hh:mm:ss date fields, and do not accept strings. Of course, you could make everything “created” on January 1 of that year, and so sorting will essentially be sorting by year only.

I just wanted to add that I’m glad to hear this is under development, and I hope it comes down sooner rather than later. I’m an academic, so the most useful metadata fields (author and journal, for example) couldn’t be replaced with tags.

This was first said to be under development over eight years ago… :confused:

“Under development”?? Is Devonthink 3 around the corner? :wink:

Hi, Andreas. As Christian has occasionally hinted on the forum, there are plans for a future generation. But no release date will be promised in advance. “No wine before its time!” :slight_smile:

I’m a heavy user of DEVONthink Pro Office for managing many tens of thousands of documents. For me, DEVONthink excels any document management application in allowing me to analyze and access the information content of my document collections. And as a “document manager” it can handle larger collections of documents than any other document manager for the Mac.

Except for relatively simple computations in Sheets, DEVONthink doesn’t “do” numbers. But of course it can hold searchable documents created using Numbers or Excel, that can be used for number-crunching of data within a database. I routinely do that in my databases that are benefited by number-crunching, such as my financial database that also deals with collecting and documenting data for tax-filing purposes.

Unlike some apps such as Evernote that change some document filetypes to a proprietary filetype, DEVONthink retains all documents in their native filetypes. It would, of course, be easier to add user-definable fields to documents via a proprietary filetype, but DEVONthink doesn’t do that – and I consider DEVONthink’s approach a major virtue. I wouldn’t want my Pages or Excel documents, for example, to be transformed into some weird new filetype that could only be opened by DEVONthink.

That means that DEVONthink’s ability to add metadata to documents is “limited” to the kinds of metadata that can be applied to any file, such as Spotlight Comments, or to those that are accepted by some filetypes, such as the Document Properties fields in PDFs or RTFs, text fields and annotations in PDF and of course metadata available for all documents via group organization, tagging, and labels within a database. Content added to a database from the Web will generally include the source URL. Scripts can be devised to change data metadata, or to add a prefix or suffix to Names. User-definable fields are not (yet) available.

But I can still add any kind of metadata I wish to my documents. I do that via creating a rich text document that is hyperlinked to one or more other documents, in which I enter searchable information. This can be very powerful and flexible, as I can add within a note cue strings that define specific kinds of metadata that are then searchable. Suppose I wish to add the source citation to a document that’s an excerpt of another document. I can quickly create an Annotation note (via a convenient keyboard shortcut) that’s automatically linked to the referenced document, and add the citation information (and anything else I wish). As the Name of the Annotation document is the same as that of the referenced document, plus a suffix, if I sort that group by Name, the Annotation will be displayed directly under the Name of the referenced document. Better yet, a link from the referenced document to the Annotation note will be created, so that it will find the Annotation note even if it were filed in a different location in a database. Although only one Annotation note can be assigned to a document, I can reference that document (and/or its Annotation note) from any other rich text note, as well.

I’m sure I will find user-definable metadata fields useful in a future version of DEVONthink. But I’ve got a lot of power to assign searchable metadata in the current version of DEVONthink.

Hi Bill, thanks for your extensive reply. I just wanted to tease you guys a little bit. I’m fine with DT as it is, it’s running all the time, here. I currently need neither sync nor meta-data (would I want to add author, year, and stuff to every .pdf inside DT), so I’m a pretty satisfied customer. The problems described by my fellow DT users above appear to be related to the what is one of the hardest and most time-consuming things to learn as a Devonthink user: to find the appropriate role for DT vis-a-vis other software and in your workflows.

A number of competing apps don’t do this.

Why, exactly? Doesn’t the database file merely reference the original file? And the database file itself contain the extra metadata?

Perhaps this is all a case of people talking about different things. I understand you to be talking about modifying the original file so that the metadata is contained within.

I (and, I suspect, many others) are merely asking for the database file itself to contain the metadata.

From looking at the annotation text file feature, I can see that each imported file in the database has a x-devonthink-item:// hash (or whatever you call it.

Why can’t the database index reference this unique number in a table and then contain the metadata too? And then display this all of this in the program.

I know that a response might be that when exporting the files or sharing them with others, you lose the metadata and that this information will effectively be locked in to Devonthink.

But this is also the case currently for Groups and Tags. If I share a PDF with someone, they won’t see the Group or Tag attributes.

So currently functionality reflects this limitation.

Further, I can’t imagine anyone having an issue with the metadata being locked in to Devonthink if it was contained in the database index and not in the file itself. If anything, it would be good for the company by indirectly fostering brand loyalty!

I don’t see how the ‘technically difficult’ aspect of this argument works.

There is, of course, the “this is my program and I’ll do what the hell I like with it” side of the coin, however.

This is exactly the kind of behavior I despise Evernote for. It is not fostering brand loyalty, it is holding a User’s data hostage and forcing him to keep using their application and system.

We are not trying to strongarm people into using our applications. That is why we have such a robust Applescripting library, do not change file formats, etc. We know people have other apps they want this data to work with. This is not the easiest job to pull off but I think we have addressed it quite well. (Bias admitted)

Thanks for your reply, BLUEFROG, but:

  1. You already have functionality that cannot be transferred to other programs: Group and Tags. Why introduce this functionality if you were so worried about portability of information?

  2. You would be introducing new functionality, not limiting it. At present, your users can’t do anything in the way of metadata. By allowing the opportunity, you are only improving on their capabilities. Using metadata, even if confined to Devonthink, is better than no metadata at all. Do not let the perfect be the enemy of the good.

  3. The database could always be made exportable in some sort of tabulated form, which would then have users of other platforms/programs pressure the publishers of those other platforms/programs to allow importing.

But at present, there is no option to do anything.

I would have thought the critical concern is not the issue of adding metadata to your data - that is always a good thing - but how to get your data out.

You could allow for this, for example by allowing the exporting of database fields in a convenient format for import to other programs (whether those other programs allow the importing of the data will be a matter for the user and the makers of those other programs to work out) would surely be a suitable compromise.

Anyway, I realise this discussion has gone on for a while. This will be my last post on the matter.

I appreciate all the prompt replies.

Jon

As a new user, custom fields are a must, because some of us have many different types of metadata that shouldn’t really be mixed. My example (as a historian): I have 10,000s of letters, each of which has an Author, a Recipient, a Date written, a Place written from, textual content, and many keywords relating to that content (logistics, about a particular battle…). Some of these letters are published (requiring Editor, Source, Volume, Page number, Year published…). Some are from archives, requiring different bibliographic metadata. Then there’s all the other info that DT has: labels, flags…
I want to limit most of my searches by the fields above, so I don’t think I can simply use annotations linked to the original document (or Groups), unless there’s a relational querying ability I don’t know about. This requires custom metadata fields for each document (especially if we want to replicate/duplicate them into other Groups). I’ve even wondered about the kludge of spoofing email so I can use it’s From, To… fields, but don’t know if it would work.

I use tags and groups, but I also need serious metadata fields for permanent info on the documents. Searching all the text to look for connections is great, but I usually need to limit my full-text searches by more than one of the above fields as well. I want to slice my data in numerous ways, and look at subsets. I want to reserve the use of Groups for difficult-to-determine keywords (so I can use Classify & See Also to automate it, DT’s bragging point), whereas Author, Year, etc. are ‘properties’ of the document that don’t change at all, are already pre-determined, and are largely unrelated to the other keywords or to the textual content. But metadata is key for limiting search results. Author X in Archive A can talk about all sorts of different topics, which is why I want the Groups to figure out those topics for me; I already know their provenance. This info should be in metadata and not in Groups, ditto the even more ephemeral tags. I don’t need Classify to tell me that this document belongs in the Archive A group; that’s an unchanging property of the document, not a variable. I need the AI to tell me that this document (which I know is from Archive A and by Author X) is talking about logistics, or the battle of Blenheim (or both)… And I don’t want the See Also list cluttered up with all the provenance groups as well as my keyword groups, I want to get rid of the provenance groups and replace them with metadata.

Just two examples of “queries” needed (can you tell I’m coming from database land?):

  1. For every letter written by Author X, where does he mention word Y? Everybody mentions word Y, and I want to find X’s references to it. Without custom fields, how do I distinguish X as the Author and not the Recipient, or X being mentioned in the text itself? (I have 100s of individuals, 100s of events…, so using tags or groups is a rather unwieldy option, plus it would presumably have to be thrown in with my many other keywords).
  2. Sort documents by source (or date written, or even date content is referring to). Sometimes I need to search or sort by the metadata alone, not the textual content (e.g. a footnote cites a specific document and I want to see if I have it already). Using naming conventions, e.g. putting the date first in the doc name, isn’t very good: in part because I have multiple date types - date of creation vs. date published vs. date discussed, and Old Style vs. New Style calendars - and in part because that’s almost all you can see of the name in the See Also drawer. My general impression is that naming conventions are basically an indication that adequate metadata isn’t available. Naming conventions in DT especially should really be to summarize the content IMHO, since that’s what you see in See Also and search results - a whole search result list of ‘1702.11.23 Marlborough to Eugene’ documents does me little good except to go through them all. It seems like using Groups here (or adding terms in the text itself) would also add all sorts of false positives to See Also, creating ‘relationships’ based off provenance that are already known and therefore noise. If I had custom fields for that fixed metadata, sorting would be a breeze.

With complex data, you need more than just 1) tags, 2) groups, 3) text search, and 4) naming conventions. You need robust metadata.

So please please include custom fields within DT (doesn’t matter if they are exportable or not). This would make DTPO a killer app for historically-based scholars. I’d even proselytize on my blog!

PS I’ve seen various historians’ discussion of their use of DT, and it’s far inferior to what one can do with a database (what I can do with Access), with the exception of the text searching, which is great in DT. But searching just off of text is a sledgehammer, especially if it’s classifying off provenance groupings that should be metadata instead. DT could be vastly improved by adding better metadata to slice more finely.

But if someone has an existing solution to my quandary, I would love to hear it.
Apologies for the rambling post.

i disagree with a number of your comments.

  1. DEVONthink doesn’t change the filetypes of documents. But it can export the user-created metadata of group organization of group organization to the Finder. And as DEVONthink uses the OpenMeta tagging system such exported content will include tags accessible to other apps that use OpenMeta tagging.

  2. While DEVONthink doesn’t currently include user-definable “fields” that can be assigned to any document of any filetype within the database, that certainly doesn’t mean that it doesn’t include other possibilities for achieving similar purposes. I’ve been able to do very tricky things since the earliest days of DEVONthink, using features that are available. Want to associate metadata such as Author to items of any filetype? Publication Date? Journal Source? Vendor? Cost? There are multiple ways to approach such needs. They include organizational structure (which can be hierarchical), tagging (which can be hierarchical), document Names, Creation or Modification Date modification, Spotlight Comment keywords, and – importantly – the ability to associate documents by hyperlinks.

Let’s take Author, for example. I could create a hierarchical tag group named Author, and include subgroup tags in it for each author I wish to list. That means I can handle that metadata, including the ability to search for all the items so tagged in the full Search window.

Or I could create rich text documents that are assigned the Name of each author, and list in it the Item Links of each document by that author, so that clicking on any such link would take me to the desired book or article. Alternatively, I could do a search (by the name of an author in document content), or perhaps on an Author tag, select all the results and copy the results list into the clipboard. I would first past the clipboard into a plain text document, which will create a list of the document names only. That list can then be pasted into a rich text document and I can do a Lookup search by Name for any of the articles or books by that author, an “indirect link” approach.

I can create a rich text document as a template, that functions as a means of creating any “metadata fields” I wish. This can be logically equivalent to the inclusion of metadata fields in a future generation of DEVONthink, especially if the template, when filled out, uses the Name of the referenced document plus a searchable suffix. Example: the template’s Name is “ItemLink Author” (select “ItemLink” in the Name and paste the Item Link of the referenced document. The template content has two lines of text: "ItemLink: " (now paste in the Item Link or the referenced document), "AuthorName " (notice the included space) and my rating of the importance of that author in his field, "AuthorRank " to which I would assign one of three terms, Important, Mediocre, Incompetent. (I could add as many “metadata fields” as I wish to such a template.) To apply the template to an article, I link it to that document by its Item link pasted into the Name. Type or copy/paste the author’s name in the first field, and assign an importance rank using one of the three terms.

Now I can search these “metadata fields” in DEVONthink. to look for all works by John Doe, do a Content search for “AuthorName John Doe” (include the quotation marks in the search query). Perhaps some of John Doe’s works were important, others less so. In more than one way, I could identify and rank his works, in ways that could be exported outside of DEVONthink.

Between such approaches and perhaps a sprinkling of AppleScript the user can actually accomplish a wide range of tasks within the existing features of DEVONthink.

In comment on your points 2 and 3, you were too focused on a specific approach to adding metadata in your databases, and didn’t try thinking, perhaps outside the box, about other possibilities. Yes, I will welcome user-definable metadata fields across document filetypes, but I can figure out how to accomplish similar results already.

I’m a great proponent of kludges, adapting existing tools to a variety of uses. I was trained as a scientist back in the days when budgets were low, and doing bleeding edge research mean that the necessary equipment had to be built from existing stuff around the lab and the local hardware store, supermarket and Radio Shack. In the early days of molecular biology I put together a pretty sophisticated disk electrophoresis apparatus from what was lying around and a plastic container from the supermarket. They were not yet for sale, back then. And as it happened, my kludged apparatus remained superior to what became available on the shelf, for quite a while. :slight_smile:

As a new DT-user I would like to add my 2cents to this topic:

I would like to have a custom field to reference scanned documents counting upwards from 1 to infinity.

I am going paperless by scanning documents and working with them in DT. I mark them with a number and just put them all in ascending order in a box. So in case I have to access the original document I can cross-reference the documents number.

If anyone has a hint on where to put that information at the moment, I would highly appreciate it. Currently I am putting it in the title. (Was thinking about tagging it binary, so having the tags 0-1-2-4-8-…)

Using Tags for a similar purpose was recently suggested in this topic:

Re: How to you associate additional data with a document in DT?

Ah, thanks a lot! Will have a look there, then.

I’m in this exact situation (and I am currently trying to determine if DT will fulfill any of my needs as a researcher)—I haven’t found a more recent discussion of this topic, and wondered if it has been resolved?

Welcome @TwilaB

Custom metadata is supported in DEVONthink 3.
Check out Help > Documentation > Preferences > Data and Inspectors > Info pane > Custom for more information on this feature.