Custom Fields

Thanks for your reply, BLUEFROG, but:

  1. You already have functionality that cannot be transferred to other programs: Group and Tags. Why introduce this functionality if you were so worried about portability of information?

  2. You would be introducing new functionality, not limiting it. At present, your users can’t do anything in the way of metadata. By allowing the opportunity, you are only improving on their capabilities. Using metadata, even if confined to Devonthink, is better than no metadata at all. Do not let the perfect be the enemy of the good.

  3. The database could always be made exportable in some sort of tabulated form, which would then have users of other platforms/programs pressure the publishers of those other platforms/programs to allow importing.

But at present, there is no option to do anything.

I would have thought the critical concern is not the issue of adding metadata to your data - that is always a good thing - but how to get your data out.

You could allow for this, for example by allowing the exporting of database fields in a convenient format for import to other programs (whether those other programs allow the importing of the data will be a matter for the user and the makers of those other programs to work out) would surely be a suitable compromise.

Anyway, I realise this discussion has gone on for a while. This will be my last post on the matter.

I appreciate all the prompt replies.

Jon

As a new user, custom fields are a must, because some of us have many different types of metadata that shouldn’t really be mixed. My example (as a historian): I have 10,000s of letters, each of which has an Author, a Recipient, a Date written, a Place written from, textual content, and many keywords relating to that content (logistics, about a particular battle…). Some of these letters are published (requiring Editor, Source, Volume, Page number, Year published…). Some are from archives, requiring different bibliographic metadata. Then there’s all the other info that DT has: labels, flags…
I want to limit most of my searches by the fields above, so I don’t think I can simply use annotations linked to the original document (or Groups), unless there’s a relational querying ability I don’t know about. This requires custom metadata fields for each document (especially if we want to replicate/duplicate them into other Groups). I’ve even wondered about the kludge of spoofing email so I can use it’s From, To… fields, but don’t know if it would work.

I use tags and groups, but I also need serious metadata fields for permanent info on the documents. Searching all the text to look for connections is great, but I usually need to limit my full-text searches by more than one of the above fields as well. I want to slice my data in numerous ways, and look at subsets. I want to reserve the use of Groups for difficult-to-determine keywords (so I can use Classify & See Also to automate it, DT’s bragging point), whereas Author, Year, etc. are ‘properties’ of the document that don’t change at all, are already pre-determined, and are largely unrelated to the other keywords or to the textual content. But metadata is key for limiting search results. Author X in Archive A can talk about all sorts of different topics, which is why I want the Groups to figure out those topics for me; I already know their provenance. This info should be in metadata and not in Groups, ditto the even more ephemeral tags. I don’t need Classify to tell me that this document belongs in the Archive A group; that’s an unchanging property of the document, not a variable. I need the AI to tell me that this document (which I know is from Archive A and by Author X) is talking about logistics, or the battle of Blenheim (or both)… And I don’t want the See Also list cluttered up with all the provenance groups as well as my keyword groups, I want to get rid of the provenance groups and replace them with metadata.

Just two examples of “queries” needed (can you tell I’m coming from database land?):

  1. For every letter written by Author X, where does he mention word Y? Everybody mentions word Y, and I want to find X’s references to it. Without custom fields, how do I distinguish X as the Author and not the Recipient, or X being mentioned in the text itself? (I have 100s of individuals, 100s of events…, so using tags or groups is a rather unwieldy option, plus it would presumably have to be thrown in with my many other keywords).
  2. Sort documents by source (or date written, or even date content is referring to). Sometimes I need to search or sort by the metadata alone, not the textual content (e.g. a footnote cites a specific document and I want to see if I have it already). Using naming conventions, e.g. putting the date first in the doc name, isn’t very good: in part because I have multiple date types - date of creation vs. date published vs. date discussed, and Old Style vs. New Style calendars - and in part because that’s almost all you can see of the name in the See Also drawer. My general impression is that naming conventions are basically an indication that adequate metadata isn’t available. Naming conventions in DT especially should really be to summarize the content IMHO, since that’s what you see in See Also and search results - a whole search result list of ‘1702.11.23 Marlborough to Eugene’ documents does me little good except to go through them all. It seems like using Groups here (or adding terms in the text itself) would also add all sorts of false positives to See Also, creating ‘relationships’ based off provenance that are already known and therefore noise. If I had custom fields for that fixed metadata, sorting would be a breeze.

With complex data, you need more than just 1) tags, 2) groups, 3) text search, and 4) naming conventions. You need robust metadata.

So please please include custom fields within DT (doesn’t matter if they are exportable or not). This would make DTPO a killer app for historically-based scholars. I’d even proselytize on my blog!

PS I’ve seen various historians’ discussion of their use of DT, and it’s far inferior to what one can do with a database (what I can do with Access), with the exception of the text searching, which is great in DT. But searching just off of text is a sledgehammer, especially if it’s classifying off provenance groupings that should be metadata instead. DT could be vastly improved by adding better metadata to slice more finely.

But if someone has an existing solution to my quandary, I would love to hear it.
Apologies for the rambling post.

i disagree with a number of your comments.

  1. DEVONthink doesn’t change the filetypes of documents. But it can export the user-created metadata of group organization of group organization to the Finder. And as DEVONthink uses the OpenMeta tagging system such exported content will include tags accessible to other apps that use OpenMeta tagging.

  2. While DEVONthink doesn’t currently include user-definable “fields” that can be assigned to any document of any filetype within the database, that certainly doesn’t mean that it doesn’t include other possibilities for achieving similar purposes. I’ve been able to do very tricky things since the earliest days of DEVONthink, using features that are available. Want to associate metadata such as Author to items of any filetype? Publication Date? Journal Source? Vendor? Cost? There are multiple ways to approach such needs. They include organizational structure (which can be hierarchical), tagging (which can be hierarchical), document Names, Creation or Modification Date modification, Spotlight Comment keywords, and – importantly – the ability to associate documents by hyperlinks.

Let’s take Author, for example. I could create a hierarchical tag group named Author, and include subgroup tags in it for each author I wish to list. That means I can handle that metadata, including the ability to search for all the items so tagged in the full Search window.

Or I could create rich text documents that are assigned the Name of each author, and list in it the Item Links of each document by that author, so that clicking on any such link would take me to the desired book or article. Alternatively, I could do a search (by the name of an author in document content), or perhaps on an Author tag, select all the results and copy the results list into the clipboard. I would first past the clipboard into a plain text document, which will create a list of the document names only. That list can then be pasted into a rich text document and I can do a Lookup search by Name for any of the articles or books by that author, an “indirect link” approach.

I can create a rich text document as a template, that functions as a means of creating any “metadata fields” I wish. This can be logically equivalent to the inclusion of metadata fields in a future generation of DEVONthink, especially if the template, when filled out, uses the Name of the referenced document plus a searchable suffix. Example: the template’s Name is “ItemLink Author” (select “ItemLink” in the Name and paste the Item Link of the referenced document. The template content has two lines of text: "ItemLink: " (now paste in the Item Link or the referenced document), "AuthorName " (notice the included space) and my rating of the importance of that author in his field, "AuthorRank " to which I would assign one of three terms, Important, Mediocre, Incompetent. (I could add as many “metadata fields” as I wish to such a template.) To apply the template to an article, I link it to that document by its Item link pasted into the Name. Type or copy/paste the author’s name in the first field, and assign an importance rank using one of the three terms.

Now I can search these “metadata fields” in DEVONthink. to look for all works by John Doe, do a Content search for “AuthorName John Doe” (include the quotation marks in the search query). Perhaps some of John Doe’s works were important, others less so. In more than one way, I could identify and rank his works, in ways that could be exported outside of DEVONthink.

Between such approaches and perhaps a sprinkling of AppleScript the user can actually accomplish a wide range of tasks within the existing features of DEVONthink.

In comment on your points 2 and 3, you were too focused on a specific approach to adding metadata in your databases, and didn’t try thinking, perhaps outside the box, about other possibilities. Yes, I will welcome user-definable metadata fields across document filetypes, but I can figure out how to accomplish similar results already.

I’m a great proponent of kludges, adapting existing tools to a variety of uses. I was trained as a scientist back in the days when budgets were low, and doing bleeding edge research mean that the necessary equipment had to be built from existing stuff around the lab and the local hardware store, supermarket and Radio Shack. In the early days of molecular biology I put together a pretty sophisticated disk electrophoresis apparatus from what was lying around and a plastic container from the supermarket. They were not yet for sale, back then. And as it happened, my kludged apparatus remained superior to what became available on the shelf, for quite a while. :slight_smile:

As a new DT-user I would like to add my 2cents to this topic:

I would like to have a custom field to reference scanned documents counting upwards from 1 to infinity.

I am going paperless by scanning documents and working with them in DT. I mark them with a number and just put them all in ascending order in a box. So in case I have to access the original document I can cross-reference the documents number.

If anyone has a hint on where to put that information at the moment, I would highly appreciate it. Currently I am putting it in the title. (Was thinking about tagging it binary, so having the tags 0-1-2-4-8-…)

Using Tags for a similar purpose was recently suggested in this topic:

Re: How to you associate additional data with a document in DT?

Ah, thanks a lot! Will have a look there, then.

I’m in this exact situation (and I am currently trying to determine if DT will fulfill any of my needs as a researcher)—I haven’t found a more recent discussion of this topic, and wondered if it has been resolved?

Welcome @TwilaB

Custom metadata is supported in DEVONthink 3.
Check out Help > Documentation > Preferences > Data and Inspectors > Info pane > Custom for more information on this feature.