Integration with Skim

Very interesting thread.

As for Skim and DTP2 : I don’t see a really major problem since skim “files” are still recognizable as PDF by DTP. To open the Skim annotations, it can be done externally …

However, this issue makes me think about a general issue that I’ve had previously realized. I think that there are a lot of expectations and feature needs in the DTP community of users, that must be anwered. It should be done by either a release of DTP or by an announcement saying that some features won’t be implemented in the near future.

As me, I think that a lot of DTP users, are “waiting” for some “little” additions to the software. They coud be well portrayed by a word said in this thread : “maturity”. I’m not saying that DTP is not mature. I’m saying that the users have the “impression” of DTP losing maturity, or not achieving modernity. We can get done a lot of work using current tools, as said in the thread, but, anyhow, we feel that there is something missing. Sometimes those “things done” are done by using little hacks, like scripts or “kludges”, not by DTP features. Yes, the work is done, but the same impression is still there, that the more we advance in our work, the more we must adapt to those and more little hacks. And when we see innovation and modernity in other apps going on (as Skim) I think that we feel that DTP is lacking something.

I think that Devon Technologies should listen to that affective kind of input. There are always rational responses, as the very smart answers of some support members. But I think that more qualitative answers must be given. DTP is not only a database, it is a database placed in a Mac environment, where there is always a lot of innovation. I hope Devon Technologies will provide more innovation and real features in the future.

Radii0

You make some good points about the development of DevonThink.

It does indeed seem to be lagging very much, and while I don’t agree entirely with some of the posts in other places (e.g. VersionTracker reviews versiontracker.com/dyn/moreinfo/macosx/31508), I do fear that an application I greatly depend on is losing ‘maturity’ and heading for obsolescence.

Perhaps you should consider also posting this in the feedback thread of the forum. I hope it galvanises other users, and the developers.

Inkwell

If I’ve read posts in the forum correctly the developers are already aware of these opinions and are probably working as fast as they can.

Thanks, jem.

I too have read the posts in the forum and would agree that the developers are already aware of these opinions. Fair enough. Beyond that, however, probability will always be trumped by certainty. And velocity of course is relative – we’re all entitled to have an opinion on how fast things are moving surely?

Of course, but I don’t think it will affect the time it takes to write the code.

Of course, assuming code is being written.

To me it has been pretty clear that this is happening.

My personal experience is that large changes to the “core” can take a lot of time but it’s not possible to show these changes since they don’t work until it’s completed. And from what I understand v2 have these large changes so I’m not surprised.

I have really fallen in love with Skim for working with pdf and would love to see DTP handle its bundle files. DTP is great for finding and relating information, but it can’t compete with an application designed specifically for annotating pdf files (nor should it be expected to). I believe I have read that 2.0 will take advantage of the annotation capabilities offered in Leopard, but These are simply bundles that contain (among other things) a the actual pdf, a txt file of its actual text content, and both rtf and txt files of all annotations. I would think that implementing support would not be terribly difficult as it “merely” requires recognizing .pdfd files as bundles with a normal pdf of the same name inside (as opposed to actually reading some other file format).

Also, I think that some of the criticisms of Skim’s proprietary file format in this thread are a bit unwarranted, given that the PDF bundle option provides annotations in rtf within a bundle.

(btw, why did a search for “Skim” not reveal this thread? I ended up starting a new thread before I happened to stumble upon this one manually)

It seems that the search index of phpBB3 is less robust than that of phpBB2. I have just started a complete rebuild and hope this fixes search problems here.

An interesting bit: Papers (application from Mekentosj) developers are considering adding full Skim support into a future release of the app. This is interesting because 1) Papers is in a some way a competitor of Bibdesk, an application from the same developers of Skim (they do not fear supporting a competitor app), 2) Papers is a really innovating application as also Skim is (they do not fear integration with other innovating solutions), and 3) most of Papers users read their PDF’s using Skim, showing that Skim is becoming widely accepted.

As for my suggestions, I hope really that the roadmap of DT2 includes my oppinions and other suggestions from many other users. We are wating, so we want to be greatly surprised when DT2 will be launched.

This thread is very interesting, especially for someone (like me) involved in academia (Humanities & Social Sciences), who is trying not to print off every journal article which comes his way… [partly for the environment, more for cost & personal inconvenience]

I’ve been a total convert to Skim, especially in its recent (stable) incarnations. I must admit I’ve been unsure what to adopt as a strategy [in-pdf comments, export to pdf etc.] so I’ve just been saving as bundles [so I all the source material that I can reprocess in Skim if necessary] AND exporting notes to rtf. I generally either import these straight into DT or process them in OmniOutliner and then export to DT from there.

As has been mentioned [at least] once earlier in this thread, a (?the) great feature of Skim is the ability to highlight & create a personal summary simultaneously. So much so that I have started exporting docs to pdf just so I can use the feature. [pity it can’t do magic for citations at the bottom of the page…]

Regardless of whether DT future includes tighter integration with Skim, it would be great to see functionality where you could open a pdf/rtf/… in DT, highlight text, and have it also copy the specific text to another document (i.e. the highlight-summarise functionality), whilst preserving some semblence of formatting. I have a hunch that this should be perfectly possible with a script (at least for RTF’s), although I must admit it is a little beyond me at the moment. Any hints would be greatly appreciated.

In my dreams I imagine highlighting text in Skim, having it appear in Omni in real time so that I can shape the structure of the notes I am exporting, and then having that integrate straight into DT.

While I am a registered user of DTPO I have since dropped it in favor of EagleFiler specifically due to Skim integration among some other capabilities. My entire reference library is in PDF format, skim annotation and therefore knowledge base integration is essential. Unfortunately the solutions or workarounds I’m reading in this thread are dodgy at best. I’ll gladly reeval DT2 but I’ve been nothing short of convinced that research essential features like this are being shelved with priorities driven on getting things like chat logs imported successfully into DT… because we all know the best research we do is wasting time on AIM and Second Life. Are users actually requesting such inane features?

While it’s true you catch more flies with honey, I’m kind of beyond that with pattern recognition kicking in from a cliche request/response record either saying wait for some distant future which isn’t representative of why I’d use a product now, or attempting to convince me why a kludgy workaround or none at all is an acceptable alternative.

What are the prospects for DT allowing the choice of Skim in a read-only mode within DEVONthink? (Maybe this issue has been implicitly raised in what others are saying, so please pardon if so.)

To elaborate: My workflow is to download a document, open it in Skim, add some notes and bring the document into DT as a PDF.

When I later want to scroll through my DT item list, DT displays the first page of each PDF. The problem is, at that point, there is no indication, without opening the document in Skim and looking at the Notes panel there, whether and on which pages I’ve added any notes to the document.

As I have a lot of PDFs in DT, it gets rather tedious to open each in Skim as one scrolls through the DT list.

It would be much more convenient if the Skim Notes panel was available within the DT window.

I asked about this on the Skim forum. Adam Maxwell’s response there was 1) Skim is BSD licensed, so the code is available, and 2) maybe DT could make use of the SkimNotesAgent, which is a background program to read Skim notes as text/RTF via a Distributed Objects connection."

An article linked by Eric B. in his blog (the blog of DEVONtechologies), suggest another workflow to follow in the case of academic research. He suggest to comment the PDF by using Skim or Preview, and then import the notes into a DEVONThink group. And that supose that you handle your PDF library with a bibliographical manager (DT works as a note repository). It can work if you are very diligent and if your workflow is always the same.

But I agree that integrating the Skim code into DT could be better, because it can interoperate with many other workflows. I agree with humanengr that having Skim a BSD licence, there are not difficulties to have a tight integration between the two programs.

That Eric’s blog refers to a website illustrating a workflow using Skim and DT is a welcome illustration that ‘the developers’ are aware of the head of steam building up from those of us who want to use Skim to read and annotate pdfs but integrate them with DT.

Two observations: first that DT has been recognised as a very good database. Second, Skim has received similar acclaim for displaying and marking up pdf’s. It is a fact, not a suggestion, that these two applications need to work together. period. No surprise then that this topic has received over 2000 viewings since December.

I think AsafKeller’s comment on 11th Dec is worth restating.

And despite Bill’s best efforts, beginning

…in the pdf world of 2008 I like humanengr, radii0, AsafKeller and clearly a lot of others want to continue with DT but seek the drag and drop, integrated, note-searchable simplicity with which say, EagleFiler, handles pdfs with Skim notes. I agree with others that Bill’s long post’s and suggested workflow just don’t cut it (whatever has worked for you, Bill, to restate, reality is there’s a popular new kid on the block, and it’s called Skim). Neither, Eric, does your suggestion answer the needs of this thread - but at least you’re listening.

Hi, Michael. Critique noted, and justified. But note that I, too, often need to annotate and search for my notes about PDF documents. And not just PDF; I want to be able to do that with any document type.

I’m a heavy user of my document databases, and I’m always working with current projects. Although I know there are good things coming down the line, I need to get things done with the tools available today.

And I do get them done, by using kludges and workarounds. Some are simple and effective, some are awkward. Whatever it takes to get the job done today. Which is why I’ll often suggest schemes that adapt the existing tools to accomplish a task.

I like the directions Skim has taken. It’s effective for annotating PDFs. But only PDFs. So Skim integration (and/or equivalent in a future version of Preview) with DEVONthink is only part of what I hope to see developing. I want to be able to make searchable notes, marginalia and so on for any file type, without unduly defacing the original document. I can do that right now, but it want to do it more easily.

I second that; this should be an easy and acceptable interim solution to allow us to effectively use Skim, which is emerging as a most popular and powerful PDF tool. (I say this at the risk of being upbraided by Bill once again; I find it interesting that The Evangelist of software designed for data storage/mining and as a knowledge base chastises users by pointing out that their predecessors were quite happy ad successful without any software solution at all, thank you very much.)

Having previously tried to underscore the importance of your posts in this thread, Asafkeller, I will (somewhat) rise in Bill’s defence because in his last he did rather ‘take it on the chin’ for which I was appreciative.

However, has anything substantive been achieved from this thread? As Radii0 said at the beginning (12th Dec)

Maybe this thread could at least alert DT to the dangers of appearing to look like something over on the dark side - a seemingly unresponsive behemoth which it is now hard to escape from. Please take some friendly advice DT. Assuming the smart answers are behind us (along with the lectures for using Skim!!) at least invest in a bit of PR. Rather than suggest users invent ‘work arounds and kludges’, you’ll earn much respect by giving a straight answer to this thread - as say the developers of EagleFiler and Papers have done. Even if you cannot match their rapid and positive response, at the very least, how about explaining your difficulties, conflicts that Skim throws up with your search routines or whatever? PR, DT, PR.

We are carefully watching this thread and may other developments out there. And we see what Skim can do. However, it is, again, using a proprietary file format that you need to actively support, with all pros and cons. For PDF annotations we are definitely going to add whatever Mac OS X 10.5 (!) holds in stock for us, and we have delayed this for so long due to backwards-compatiblity to Mac OS X 10.3. For other file formats, such as RTF or whatever, adding annotations means creating another proprietary file format on our own including a hell of problems.

So, even though some here don’t believe that, we have current completely disassembled (and party re-assembled) DEVONthink for implementing a new underpinning that will solve many problems and restrictions that results from its growth in the last five years (DEVONthink is today far from what we had in mind when this project started.) And when these changes are done we will implement the top most requested features, number 1 one being the ability to open multiple databases at a time, number 2 being more file formats, and number 3 being a modernized user interface. And, this focus (including having focused for a while on a new, still to-be-discloses product (!)) is the reason why we’re not adding things like Skim support to DEVONthink 1.x anymore.

Please watch my blog, I’ll definitely keep you up-to-date with everything that we think we can share as soon as we can share it without putting ourselves in the danger of torpedoeing both our projected schedule and your expectations.

I don’t think that dis-/reassembling DEVONthink’s code is a party :slight_smile: