Indexing set up (again) - could you help me think this through, please?

DEVONthink
1

Hi, I’ve been digging around the forums but am still trying to work out what best to do - and some of the responses are quite old now so I’m not sure what’s up to date advice.

My situation: I’m studying again (mature student, part time). Right now I don’t really need to make use of a reference manager / bibliography, but may well need to in the future. I have fully licensed DTP, but am really only using it as a repository of PDFs, do the occasional search (more of that in the future, I suspect) and to OCR scanned documents (ca 5% of my documents). I have a few hundred (possibly 800?) PDFs, some that are core reading and some that I’ve collected as further reference.

I’m a very heavy iPad user (80% of my study time) and want to annotate PDFs using hand written annotations rather than text based highlighting. I currently don’t have much of a need for exporting annotations - I’m actually trying to replicate a pen on paper approach so am using split screen on iPad to annotate PDF on one side and jot bullet notes of what I’m reading in a notebook with the Apple pencil on the other side. Occasionally, though, I’ll select text in the PDF and drag and drop it into my notebook - hence why the OCR recognition is useful.

However, the annotation in DTTG isn’t so great - no dark mode (I do quite a bit of reading in bed and don’t want to disturb my husband!) and no bookmarking. Bookends annotation looks similar but has more functionality. So as far as I can see, if I want to continue to use DTP, is:

1- keep all PDFs in Bookends for a) future proofing of referencing and b) better annotation, and index into DTP. BUT I can’t get Bookends syncing to work smoothly across my two iPads and the MacBook, not helped by only having a trial Bookends licence which means I don’t have mac cloud sync (and I don’t currently have £60 to spend on a software set up I’m not fully sure I’ll adopt)

2- keep all documents in dropbox / iCloud and index into DTP. This has the advantage of being able to use, say, PDF Expert to annotate but I’m concerned about the fact DTP loses the connection to the index filed if you rename in finder.

3- (wildcard)- keep all documents in the margin note folder, as this app does both annotation and OCR. But I don’t take to the app very much, I find it more distracting than a simple annotation tool.

I’m also a bit concerned about all the scripts people use to get things working between DTP and BE. I’m not at all used to scripts or how to run them.

So, to summarise, my requirements are:

  1. Be able to ‘park’ documents in folders (either in DTP, or in Finder) and annotate them from there rather than to import them into apps (as you have to do with liquidtext / margin note)
  2. Be able to use a better annotation tool than DTTG, such as Bookends or PDF expert.
  3. Be able to use OCR functionality to recognise PDFs on the relatively small amount of docs that are scanned.
  4. Not have to worry too much about mucking up the set up if I rename a document in finder.

Any thoughts please?

2

At least dark mode will be supported by the next release.

3

Oh, really? That’s EXCELLENT news, that might be enough to make it workable for me. Any idea when that release will be?

4

We don’t announce release dates but definitely this spring.

5

Okay, that’s fantastic. I’ll stick with my current set up in DTP then, thank you.

6

Note… if you have purchased the Pro Pack in DEVONthink To Go (see Settings > Manage), you can access files in your databases in other applications supporting the document provider mechanism.

PDF Expert does. Click the Files option on the left side of PDF Expert and you should be able to browse the databases.

7

A few thoughts… Sometimes I like to get started by writing on my iPad, too. I write in Goodnotes 5 side-by-side with PDF Expert. Sometimes I’ve highlighted in PDF-ex and write notes on those. But my handwriting is just another way of thinking, and I often shift to using the iPad as a monitor and type more permanent notes. Sometimes I get an idea that I need to write down or about, and I do that in Ulysses. It’s almost as instant as paper and it’s with me everywhere.

However, If I’m typing an “archival note,” I type it into DT3 in the Inspector/Reminders & Annotations/Annotations box. I love this. The annotations can be linked to the pdf; other documents (cf., xxx) can be linked to pdf and annotation. You can make different annotation templates, for different functions, too. The comments box, right above the annotations box is also great: I sometimes use if for checklists, e.g.," [ ] compare P.2 with E.Z Theory." And I can set a timer to make sure that I return to the entry.

I find reference managers essential (and have successful colleagues that merely keep a running list of references in TextEdit) and imperfect. Right now I use Bookends. It renames pdfs into a format (e.g. author, year, title) that you find most readable, and it ‘parks’ PDFs in an Attachment file, which can live in Dropbox. I’ve never been tempted to rename the Attachment File. One reason is that it’s so easy to group references, and many ways of doing it, in Bookends. (Also, Scrivener and DT3) The file name itself turns out not
to be so important compared to what you’re doing with it. And the same file can be treated, named, grouped differently without moving of changing the file document. But I do understand the need for a name!

DT3 can index the Attachment file; Scrivener can link to it. Now if a pdf needs OCR, you could just import it with “Images (with OCR)…” and export to BookEnds and its Attachment file if you want.

The most important job of the reference manager, to me, is renaming files (otherwise, a bore) and putting together the bibliography for publishable work in the right style with the correct details. If it weren’t for the renaming of the pdfs, I probably wouldn’t keep pdfs in the reference manager at all. The temptation would be to import all the pdf files into DT3, where they could live in a long lists except the occasional groups that are papers from a single issue of a journal (issues which can also be handled by tagging).

I don’t use Scripts, but am glad for those who do. I was helped by a book, Digital Paper: A Manual for Research and Writing with Library and Internet Materials by Andrew Abbott. It’s not for everyone. It made me recall that the bottleneck in academic work is the human brain. You can use it for your work, or use it for your apps. Some kinds of research in the humanities and social sciences are highly recursive, and DT3 facilitates that kind of recursion, enabling looking at your material from different angles.

Everyone has different ways of addressing their questions and the different problems to solve. I don’t know the ins and outs of the apps that I use–I skim off the top what they do best. And decide what I’ll use depending on the friction in my thinking and which oil works the best in the moment.

Good luck in your studies!

Linn

1 Like
8

Doh!! Yes, of course, thank you. I even used to use that a while ago.

Assuming I decide not to store PDFS in bookends, this the answer, I think. Thank you.

9

This is incredibly useful, thank you.

Because I’m so ipad focused, these aspects of DT3 (annotations, etc) are ones I’ve just not explored and they sound extremely useful.

Your work flow points me back again at storing PDFS in bookends and indexing. I appreciate your caveat that you only skim the possibilities of these apps, but if you were in my position, having (say) 800 pdfs currently sitting in Dropbox (well, actually, imported into DT3 a couple of days ago) of which, say, 50 are related to the studying you are actually doing, and of that 50 maybe 15 are ‘core’ reference material, how would you approach a migration of files to bookends?

As far as I can see, bookends doesn’t support subfolders so I can’t (unlike when moving into devonthink) simply drag and drop what I have. My understanding is at the minimum I’ll need to create groups and assign selected pdfs to those. And of course there’s the job of editing references on each.

So the thought of bulk importing 800 pdfs seems or of the question. If, however, I import just the ones I use, as and when I use them, I’ll end up with files spread across the two systems. Any thoughts, please?

10

No problem

11

You’re welcome, @JKF. I think that you might be surprised and interested in what DT3 offers for taking notes. Try the following to give you an idea. You’ll need to install the DT3 clipper extension (from their website) on your Mac.

Select a web page in your browser (even this one), clip it with the DT3 clipper extension, type a little note in the COMMENT field , e.g., “I clipped this site because…” and then press ADD (bottom right). Then go to the database you had opened when you started (or the one in the Location field of the clipping. Find your clipping and Press ^3 if you don’t see the Reminders and Annotations Inspector. I’m still studying the Inspectors workings, but this little exercised inspired me to look more closely into DT3 as a note taker for the first time.

There is no “migration” between DT3 and Bookends. To move files back and forth between them, just use the widespread menu commands, File/Export or Import. In DT3, select all the pdfs you want to export and click on the “File/Export/Folder and files…” to copy your material to the Finder. From there, you can go to Bookends and do a File/Import. Bookends add the references to the Attachment Folder (or creates one if there isn’t one). You already have all your 800 files in DT3, so basically, you’re done.

You raised some concerns about your files “spread across two systems.” You don’t need to keep the two systems “the same.” They are fundamentally complementary. Bookends does one thing exceptionally well: reference managing. DT3 helps you do almost everything else: your studies, your papers, your other creative activities. You also may have concerns about using too much space on your computer by duplicating pdfs. This can an issue for large images and movies, but pdfs are much smaller.

A question for you: I don’t understand the 800 folders. What motivated you to collect them? How do they fit into your project when you already have a handful with the 50 documents vying for your attention?

12

Thank you, that’s really helpful.

I do virtually no research or learning in my Mac, I’m entirely ipad based with Mac only for essay writing. I am thinking I need to change that balance to harness the additional functionality.

You are off course correct about the 800 pdfs. I was thinking very much all or nothing: that they should all exist in DT or all exist in BE. I guess what you’re saying is I might only take into BE the ones I’m using for a project. That’s obvious, really - I was over complicating it :joy::joy:

As for having collected them - again, a very good challenge. I went through a stage of collecting information rather than processing it - rooky error!! But now I have the material, so I need to find some way to organise it so it’s not distracting.

13

Just put the 800 folders into a database of their own for now. They’ll come to you when you need them.

Good luck with your studies!

Linn

14

@JKF sorry to revive this old thread, but is it not possible to have all PDFs in a MacOS folder (e.g. the “Preview” folder) and have a DT database indexed to it, and Bookends watch it? In that case, would annotations in both apps modify the same file?