peculiarities with synchronizing

DEVONthink Feedback
1

I’m noticing that when I index a folder and ask it to synchronize, DTP 1.2 doesn’t exhibit the behavior I would think it should. I’ll mention two issues that drive me up the wall and see if this is a bug (as I think it is), or that it’s just such a strange way of doing things that I fail to understand why it would be useful. Basically, I’ve set up a folder in my finder that has an important set of bookmarks that I use for my research. I click on the link I want to go to, it goes there, and I’m happy.

But strange things start to happen under two circumstances. If I add a new bookmark to my items to synchronize, I would expect dtp, on issuing the synchronize command, to export that “link” file out to the finder folder. But it doesn’t. Am I wrong in thinking that synchronization should be a two-way synchronization? Wasn’t this part of 1.2? Linguistically, i guess I’d say that synchronize usually means two way…and if it is only one way File system–>DTP why not call the item “Update from Filesystem”? Oddly, I find, if I click on the item that has been dragged into the sync folder, go to the menu and select “synchronize,” it will let me, but it won’t synchronize the file, doesn’t put the little blue tilde next to it, doesn’t seem to do a thing. Shouldn’t it synchronize to disk? And if not, it seems it should not let me choose synchronize on an individual item.

I of course would really, really like the synchronization to be two-way to the file system, because otherwise, if I want to add that bookmark or a file, I have to export the whole folder again and reindex it, which is not only a lot of work, but error prone, especially if the (file system) folder is shared among other databases in dtpro. I assume everyone else would prefer some way to do a two-way sync as well, right?

The other thing I notice is similiar but I assume simply a bug. If I index a folder that contains other folders, including some empty ones, if I delete a folder that contains files and then tell dtp to “delete on file system as well”, the folder and its contents will disappear on the next sync. But if I click on an empty folder and delete it, dtp will delete the folder without asking. But then when I sync, it will simply reappear. The only way to get rid of it is to go to the file system and delete it. That seems wrong, too.

I know I’m griping here, but I guess I think that the synchronization thing would be really nice if it worked two-way, especially for the small subset of files that I use between all of my databases. I would like to have that set automatically synchronize with a script and be available no matter which database I’m in. This would be so wonderful, and would save silly data redundancies. But I of course want to be able to update that data from time to time, and have DTP simply update it to or from file system as appropriate. I know that great file system exposure might be part of the plans for an (ahem) later version of the software, but if there were any way in the 1.x series to at least be able to tell it manually to change a file within an indexed folder from Dt-represented to indexed (i.e. file system represented), perhaps in the Info panel, this would be at least a nice alternative. :stuck_out_tongue:

Just some thoughts,

Erico

2

okay,

Digging through the help section reminded me that contrary to what I had thought, 2 way manual synchronization is not part of 1.2. So I guess that observation #1 is not a bug, but it sure is frustrating. I think the folder problem is a bug, though.

But besides adding me to the list of people strongly looking forward to some version of dtp where the syncing is two-way, even if on a limited basis, I would like to suggest it would be nice if on an interim basis there were a way to work around this with a good script.

Dtpro could do two-way sync if there were an applescript command to “index” an item in the finder, rather than importing it to the database. (In other words, just expose the “Index” menu item to the AS dictionary) Then I could write a script to detect all not indexed items in the current folder, export them to the appropriate finder folder, and then index them and delete the originals. Not perfect, but it would help with a category of usage that I suspect others would like too.

Is there any hope of squeezing an “Index [posix path] to [record id]” into dtp 1.2.x if two-way sync is a long ways off?

Just a thought…

erico

3

I am convinced there is always a way :slight_smile:

When indexing a folder, the path is set in the Group record in DT. An Applescript could iterate through the contents of the folder specified in the path (using ‘tell application Finder’). For every item in the folder, the script would skip the item if a record exists in DT, and otherwise index the item. The script would recurse into child folders and search fo new items there as well.

Written correctly, such a script could be attached to a group so that every time the group is opened, the script runs and synchronizes the group and all children.

4

yeah, I think its even closer than you suggest, and yet further away…

DTP has an “indexed” property for each record, and I see no reason not to trust it, so a script wouldn’t actually have to check every file/path with applescript calls out to the finder or to calls to the shell…you would only have to update the files that were in the DT hierarchy that are not already indexed.

This wouldn’t take forever at all. I already have code that will do the recurse without any problem, but the problem is that for an applescript that would allow for two-way scripting there would need to be script command that allows one to index a single file and put that index record in a given known folder, i.e. to “add” an index item to a destination. Otherwise I don’t think there is a way…

There is of course the funny issue that in my opinion an “indexed folder” is a special kind of thing, one which I would think would imply automatic export of any item dragged into it out the file system. But that’s not the way dtpro works right now. As I said, a way to manually index a single item to a destination child folder would be a really nice addition the wonderful dt applescript dictionary. hint hint… :smiley: :smiley: :smiley:

erico

5

Not quite sure what you mean.

To sync a record from DT to the file system (export), you can use the ‘export’ command to create an output file of the approrpiate type, then create a new record for the exported file (setting the Path correctly and setting Indexed to True). At this point you should remove the original record.

To sync a record from the file system to DT (import), you only do the second step: make an index record.

Oh wait, I think I understand – you want DT to handle syncing from the filesystem to DT, but to have the script sync from DT to the filesystem. Right? The only problem being that DT will not (or so I have read; haven’t tried it) import newly-created files from the filesystem when doing a sync, so your script will have to compare the contents of the folder and the group to bring new files into DT.

6

Actually there’s an “index” script command but it’s called “indicate” (“index” is a term already used by AppleScript and AppleScript doesn’t like duplicate terms).

But it might be even easier:

First, index a folder (e.g. by command-option dragging & dropping it to DT Pro). Then save the following script and attach it to the indexed group:


on triggered(theRecord)
	try
		tell application "DEVONthink Pro"
			synchronize record theRecord
		end tell
	end try
end triggered

Opening the group or selecting it in split/3-pane view will execute the script and automatically sync the group. It’s of course unidirectional but useful to map folders to DT Pro’s database.

7

yes, exactly, this is what I was thinking, because as far as I can tell (I did just try it), dtpro will find a new file in the file system and index it if you tell it to synchronize the file.

But no, I don’t see a way to do this, because the index property is read only.
But do you see some way to make this work:



tell application "Devonthink Pro" 
set group_id to current group
make index record with path "/tmp/" in record group_id

end tell

My sense is that dtpro won’t allow this.

8

Christian,

oops, my reply came after yours. Thanks for indicating that indicate actually exists, I had missed it…

I actually have been doing the attach script you mention, and love it, but then wanted to figure out a way to do things in a bi-directional way if I drag things from an existing (non indexed) dt folder to an indexed one. (I keep wanting to be able to put a “on drop items” trigger script on a folder, but that’s another story)

But okay, thanks to your help, I now see that it would be pretty easy to have at least a one folder level deep bi-directional sync script, and only somewhat more difficult to have a multiple layer one…because I could just figure out the non-indexed files and either flush them out to the file system with an export, delete them, and then invoke a sync, or I could simply export them and then “indicate” them. groovy…runs back to script debugger…will post here when done.

once again christian, I keep being surprised at what is in that amazing as dictionary of dtpro. It is just tremendous.

edf,

thanks for pushing me to think about it a bit more…I obviously hadn’t looked hard enough at the newer as dictionary. Will report back after trying this out a bit.

-erico

9

Thank you both…I just hacked together this little script that does what I want at least for now…bidirectional syncing on an already indexed folder. You can now drag things into that indexed folder from other places in devonthink and it will export those files to the file system appropriately & import files from the file system as appropriate.

Give it a try but be warned…it has no error checking…

I’m interested in your criticisms…

Eric



----index the unindexed  by eric oberle
----a first try at a bi-directional sync update for devonthink pro
---warning: has no error checking


tell application "DEVONthink Pro"
	set current_rec to the selection
	repeat with a_rec in current_rec
		
		my index_theunindexed(a_rec)
		with timeout of 500 seconds
			synchronize record a_rec
		end timeout
	end repeat
end tell


on index_theunindexed(top_rec)
	tell application "DEVONthink Pro"
		set the_children to get children of top_rec
		repeat with this_child in the_children
			if type of this_child is group then my index_theunindexed(this_child)
			if indexed of this_child is false then
				set import_path to name of this_child
				set export_path to (path of top_rec)
				export record this_child to export_path
				delete record this_child in top_rec
				set import_path to export_path & "/" & import_path
				indicate import_path to top_rec
			end if
			
		end repeat
	end tell
end index_theunindexed
10

Testing this seems to cause no harm other than leaving extra metadata files (devon-tech.storage files) in the directory where updates occur. I’ll tinker with this a bit more, put some error checking in and post it to the applescript forum. Meanwhile everyone who doesn’t have a backup of their data should stay far away from this…

Thanks again, this really makes my life a lot easier, and makes having multiple databases more practical. It isn’t even that slow if run from the script menu or as a triggered script…

-eric

Addendum:

There is an experimental and improved version of this script in a different thread in the Applescript forum:

devon-technologies.com/phpBB … 6048#16048

11

Christian,

Working a bit more with the synchronizing routine, I guess I now see four things that still seem a bit odd to me. I don’t know if they are bugs or just philosophical/design issues, but I thought I’d mention them in this thread, because maybe they are things I’m just not understanding.

  1. as above, if you use DTpro to delete an empty subfolder of an indexed folder, it will not prompt you to delete the files on disk, but then when you perform a synchronize command it will bring that folder back.

  2. It seems like if a folder is indexed, and one renames it, DTPro should rename the folder in the file system and change all paths in the subtree below accordingly.

  3. Philosophically, I think point 2 should apply to all file names as well…in other words, if a file a member of a group that is indexed to a finder file and I change the name of the file item in DTPro, shouldn’t it change the file name in the finder as well.

It seems to me that dtpro should just do the right thing on the name changes for 2&3 in question, but it seems to me that it should certainly do it when the synchronize command is invoked. But it does not at the moment. It would be nice if this were an option. Or maybe a button in the get info panel to “update filename in filesystem”?

  1. It also seems to me that the synchronize command should grey out when one has neither selected an indexed file or a group that contains one.

I’m sure there might be bigger issues behind these peculiarities***, but it seems to me that they could be changed without breaking any other behavior. It also seems to me that if the folder-level issue isn’t changed, it is extremely likely that external files will eventually need to be en masse exported & reimported because they will get out of sync with the file system.

These issues aside, the indexing/syncing feature is really neat. It sure makes my file system backups go a lot more quickly!

best,

eric

****I can only think of one serious consequence of changing things the way I am suggesting, which is that when one moves an indexed item out of an enclosing indexed folder (i.e. when one moves a child of an indexed parent to a non-indexed parent) the thus moved files should probably be reimported into the dt database. Obviously, one should be able to replicate items however one wishes without reimporting.

12
  1. That has been requested, some people didn’t like DT Pro deleting empty folders
    2./3. V2 will probably work that way
  2. Synchronize supports also imported files/folders
13