Grouping items by date opened or like a daily journal

I’m finding it challenging to find and recall much of the writing and reading that I do. I recently purchased DEVONthink with the hopes that it would help me stay organised. So far I have had little luck.

I’m a freelance knowledge worker with a background in media studies. I find it impossible to distinguish between the “work” I do for one gig or the other, or separate that from my own personal interests.

I’m constantly downloading PDF files and bookmarking websites, and constantly writing notes to myself.

Before using DEVONthink, I would often create aliases/symbolic links with macOS Finder in order to store my academic reading in multiple locations simultaneously. DEVONthink seems like a better solution with its Replicant function.


I have an idea for an organsitional system, but I’m not sure how to implement it. I want to be able to easily keep track of the work I do every day. I’ve thought about a couple possible solutions, which I’ll describe below.

  • Tag everything I do with today’s date. Every time I open a file or create a new file, it would get a new tag added. It could take many possible syntax, but I would have to commit to one and stick to it. ie. #2021/October/27, or #20211027, or #2021-10-27, etc.
    • Cons: cluttered tags
    • Pros: ability to easily identify all the days that I’ve interacted with a particular item.
  • Create a new group every day. Replicate all the items that I interact with on that day into the group.
    • Con: I don’t know how to implement this in practice, except by hand, which ends up being incomplete.
    • Pro: easily see the progression of my ideas from day to day in sequence.
  • Every day, write a note where I list all the items I engage with and include item links.
    • Con: time consuming and difficult to enforce completely. I end up with many incomplete and partial daily entries. Maybe automation could help?
    • Pro: adds a narrative to the data and connects items through storytelling

I don’t know what to do. Can anyone help me work through this problem? I think automation via smart rules or smart groups or scripting will play a part in whatever solution I pursue because I don’t have enough personal discipline (or spare time) to enforce this sort of system on my own. I probably will land on some solution that takes inspiration from a combination of the three methods I outlined above, or maybe something entirely different.

I use an Applescript to process items in my inbox
It could easily add/assign the date tag

Personally, I prefix note names with the date instead of date groups/tags
My naming standard is subject-date type [description] keywords
for example 2021-10-01 Receipt [Lunch] !Vendor-aaa !Budget-bbb $3.50
An Applescript assists with this name process

My problem with naming items by date is that I often work on the same document over the course of many days.

I guess I could duplicate the document and rename it every day I work on it. In that scenario, naming would act a bit like version tracking. But I worry it would clutter my search results with duplicates…

We can trigger a smart rule every time a note is saved

How about a journal note which is a list of all work done during the day
A script can append entries throughout the day
The entries can be a links to actual notes

2 Likes

I like that idea! How would I go about scripting that? I’m a novice although I’ve been trying to teach myself. I have no idea where to even start for something like this. Usually I find an existing script and tweak it to my needs or combine it with parts of other scripts that I find online.

I’m picturing it a bit like this: Whenever I save a document for the first time that day, a pop-up appears with the prompt “Describe the way you’re using this item.” I write a short sentence or two to provide context for when I review this journal entry sometime down the line. Then the link to the item and my description appear as a new line in my daily markdown journal. At the pop-up, I would also have the option to cancel, which would prevent that item from becoming a new entry in my journal. And the pop-up should include a “choose file” dialog where I could select additional items to include or even the group enclosing the item I was working on, overriding the original hyperlink and instead entering these items in its place.

1 Like

This Smart Rule script creates a Markdown record for each day with links for all opened records.

The record is created in group Record History in the global inbox.

Properties

  • property sortByLastOpened : Set to false if you want to see when a record was first opened.
  • property theExcludedDatabaseNames : List of database names that should be excluded.
    E.g. a test database or a RSS feeds database are candidates for exclusion.

Setup

  • Create a Smart Rule:

    • Criteria: kind:any
    • Event: On Open
  • Exclude the group from classification (Tools > Inspector > General)

Edit: 2021-11-12 updated script

-- Smart Rule - Create Record History

property sortByLastOpened : true
property theExcludedDatabaseNames : {"_temp"}

on performSmartRule(theRecords)
	tell application id "DNtp"
		try
			repeat with thisRecord in theRecords
				if (name of database of thisRecord) is not in theExcludedDatabaseNames then
					
					set theHistoryRecord_Name to my formatDate(current date) & space & "Record History"
					set theResults to search "kind:markdown {any: name==" & theHistoryRecord_Name & " name==" & theHistoryRecord_Name & ".md}"
					
					if theResults ≠ {} then
						set theHistoryRecord to item 1 of theResults
						set theHistoryRecord_Text to plain text of theHistoryRecord
					else
						set theGroup to create location "/Record History" in inbox
						set theHistoryRecord_Text to "<style> a {text-decoration: none;} </style>" & linefeed & linefeed & "# " & theHistoryRecord_Name & linefeed
						set theHistoryRecord to create record with {name:theHistoryRecord_Name, type:markdown, plain text:theHistoryRecord_Text, exclude from see also:true, unread:false} in theGroup
					end if
					
					set thisRecord_ReferenceURL to reference URL of thisRecord
					
					if thisRecord_ReferenceURL ≠ (reference URL of theHistoryRecord) then
						set thisMarkdownLink to ("[" & my escapeLinkName(name of thisRecord) & "](" & thisRecord_ReferenceURL & ")") as string
						if theHistoryRecord_Text does not contain thisRecord_ReferenceURL then
							set plain text of theHistoryRecord to theHistoryRecord_Text & linefeed & "* " & thisMarkdownLink & space & space
						else if sortByLastOpened then
							set plain text of theHistoryRecord to my updateOrder(theHistoryRecord_Text, thisRecord_ReferenceURL, thisMarkdownLink)
						end if
					end if
				end if
			end repeat
			
		on error error_message number error_number
			if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
			return
		end try
	end tell
end performSmartRule

on formatDate(theDate)
	set theYear to year of theDate as string
	set theMonth to ((month of theDate) as integer) as string
	if (count theMonth) < 2 then set theMonth to "0" & theMonth
	set theDay to day of theDate as string
	if (count theDay) < 2 then set theDay to "0" & theDay
	return theYear & "-" & theMonth & "-" & theDay
end formatDate

on escapeLinkName(theText)
	set d to AppleScript's text item delimiters
	repeat with thisCharacter in {"[", "]", "(", ")", ">"}
		set AppleScript's text item delimiters to {"\\" & thisCharacter, thisCharacter}
		set theTextItems to text items of theText
		set AppleScript's text item delimiters to "\\" & thisCharacter
		set theText to theTextItems as text
	end repeat
	set AppleScript's text item delimiters to d
	return theText
end escapeLinkName

on updateOrder(theText, theReferenceURL, theMarkdownLink)
	set theDelimiters to {"(" & theReferenceURL & ")" & space & space & linefeed, "(" & theReferenceURL & ")" & space & space}
	set theTextItems to my tid(theText, theDelimiters)
	set theTextItems_modified to {}
	repeat with i from 1 to (count of theTextItems)
		set thisTextItem to item i of theTextItems
		set thisTextItem_Paragraphs to paragraphs of thisTextItem
		set thisTextItem_Paragraphs_Count to (count of thisTextItem_Paragraphs)
		if thisTextItem_Paragraphs_Count > 0 then
			if item -1 in thisTextItem_Paragraphs contains "x-devonthink" then
				set theTextItems_modified to theTextItems_modified & thisTextItem_Paragraphs
			else if thisTextItem_Paragraphs_Count > 1 then
				set theTextItems_modified to theTextItems_modified & items 1 thru -2 in thisTextItem_Paragraphs
			end if
		end if
	end repeat
	return my tid(theTextItems_modified, linefeed) & linefeed & "* " & theMarkdownLink & space & space
end updateOrder

on tid(theInput, theDelimiter)
	set d to AppleScript's text item delimiters
	set AppleScript's text item delimiters to theDelimiter
	if class of theInput = text then
		set theOutput to text items of theInput
	else if class of theInput = list then
		set theOutput to theInput as text
	end if
	set AppleScript's text item delimiters to d
	return theOutput
end tid

5 Likes

Wow! Thanks for this! I’ve got it set up and running smoothly.

Now to start tweaking it….
I already added an additional trigger, so that it runs not just when an item is opened but also when a new item is imported.
I’d like to prevent it from adding duplicates. Maybe a custom metadata flag could assist with that.

Each record is only added once.

There are some exceptions, I haven’t figured it out yet. Here’s an example. I think this may have happened when I used space to quick-look before opening? or maybe I opened it in with DTP3 and with an external editor? not sure but it’s happened a few times, although not every time.

* [181899847-Subculture](x-devonthink-item://5C9191B8-43CC-470D-B800-9F37472FEA9E)  
* [181899847-Subculture](x-devonthink-item://5C9191B8-43CC-470D-B800-9F37472FEA9E)  

No, it was a silly mistake. I didn’t use a variable and instead wrote out the string that should be replaced and of course did it wrong … one time with the linefeed at the end, one time at the start. The problem only shows if one opens the last added record immediately again, i.e. if no other record was opened in between. Fixed. Thanks!

Hi pete31,
This is an awesome script. Super helpful. Thanks for sharing.

1 Like

Updated the script. Improvements:

  • Escape link names for Markdown.
  • Use only the Reference URL to sort records by last opened.
    (Previously renaming and opening a record again yielded unwanted entries)
  • Added property that can be used to exclude databases.
  • Mark Record History as read.

Edit: Updated script again.

2 Likes

Thank you for posting this. Very nifty.

1 Like

Changed my setup: Adding openingDate:Today and trigger On Rename covers cases where a record was opened before renaming it and not opened again afterwards. Without it the Record History still shows the old name.

  • Criteria:
    • kind:any
    • openingDate:Today
  • Event:
    • On Open
    • On Rename

If property sortByLastOpened is set to true then renaming a record also moves it to the bottom of the list which is not 100% correct but better than still showing the old name, I think.

1 Like

Seriously cool idea and script. I had no idea that I needed this until I tried it. So very hepful.

1 Like

For your next trick … is it possible to create a Smart Group containing all the items linked from a Record History note that this rule creates?

For instance, I’m thinking of a case where I read five different papers two weeks ago, and now need to confirm exactly which one contains a specific fact.

Same here! Thanks a lot @aaaaaaaaaaaaaaaa !!!!!!

1 Like

Yes, but unfortunately not easily. The only way I can think of is to tag the entries. But one can’t create a global Smart Group via AppleScript which means you’ll have to create it manually.

-- Assign tag to selected History Record's entries

property copyTag : true -- Copy tag (i.e. name of the History Record) for easier creation of global Smart Group

tell application id "DNtp"
	try
		set theRecords to selected records
		if theRecords = {} then error "Please select a History Record"
		set theRecord to item 1 of theRecords
		
		set theRecord_Source to source of theRecord
		set theReferenceURLs to get links of theRecord_Source
		set theRecord_NameWithoutExtension to name without extension of theRecord
		
		repeat with thisReferenceURL in theReferenceURLs
			set thisReferenceURL to thisReferenceURL as string
			set thisLinkedRecord to (get record with uuid thisReferenceURL)
			if thisLinkedRecord ≠ missing value then
				set tags of thisLinkedRecord to (tags of thisLinkedRecord) & {theRecord_NameWithoutExtension}
			end if
		end repeat
		
		-- It's not possible to create a global Smart Group via AppleScript so this needs to be done manually
		
		display dialog "Please create a global Smart Goup with query:" & linefeed & linefeed & "tags:" & theRecord_NameWithoutExtension
		if copyTag then set the clipboard to theRecord_NameWithoutExtension
		
	on error error_message number error_number
		if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
		return
	end try
end tell

1 Like

It would probably be far better to use custom meta data. I guess you don’t want the tags to stay around so you’ll have to delete them at some point (and this in each database). Custom meta data could stay attached to the records without spoiling the UI much.

Custom meta data could be attached while adding the record to the History Record. However, it would still be necessary to manually run a script as it might be that not all entries’ databases are open all the time.

Easiest one for me, create a Smart Group with all items (“Kind” equals “Any Document”). Open it, then sort by date.