Devonthink for Qualitative Analysis

DEVONthink Feedback
1

I love DTP and have found it invaluable to manage my doctoral research over the past several years. I’ve stuffed a single DB full of PDFs both big and small along with my own RTF notes and I’ve really appreciated its seemingly unlimited capacity and general zippiness.

I was perusing the previous threads about using DT for qualitative research. I noticed that on one thread the response was that you hadn’t received any requests for it lately so it wasn’t being considered as a development goal. Please don’t assume that lack of users begging for it means there’s no market. It’s frustrating how close DT comes to being useful for qualitative research. I’ve used DTP for years and I would love to use it instead of one of the expensive qualitative data analysis programs or one of the more clunky open source versions. They’re difficult to use and not very flexible. (There are very few programs that run on Macs without Windows emulation.) Please consider supporting this use in your development cycle. I think marketing DT for qualitative analysis would open up a whole new market for you making it worth the effort.

en.wikipedia.org/wiki/Computer_A … s_Software

3 Likes
2

Hi! Thanks for your post.

First of all, let me note that while I’m employed with DEVONtechnologies, that nothing I say here should be interpreted as a statement of intent. I don’t want to give anyone any false impressions :slight_smile:

I remember those threads. Certain features would, I believe, be helpful to a large subset of the userbase and may constitute a useful “path” for DEVONthink to explore as it adds some amount of QDA functionality. I like the ability to tag subsections of documents, and/or to treat multiple documents as a larger “virtual” document and a large document as a collection of smaller “virtual” documents. The capability for realtime information about data characteristics (i.e., the number of occurrences of a phrase in a specific set of documents) also appeals to me.

However, some of these features would immensely complicate the application as it currently exists (I expect that the tagging engine would have to be rewritten or massively augmented) and might compromise other goals (tagging subsections of documents in certain formats might be easier than tagging subsections of others, leading to differences between application capabilities for those formats, which is messy and hateful to everyone). And so forth.

Since DEVONthink wasn’t designed from the ground up with this functionality in mind, and has evolved considerably since QDA was seriously considered, the complexity of implementing even small parts of the necessary feature set may be prohibitively large.

If you don’t mind, could you list some more of the features that would enable DT to be used for QDA? That would help us to consider the proposed feature set.

To conclude, I have no “pull” in this matter and can’t guarantee anything, but I would like to see DEVONthink grow and be useful to more people. If nothing else, perhaps I can pass some suggestions up the chain to the DT developers to make DT more useful, or at least acceptable, to people wishing to use it for QDA :slight_smile:

Thanks again for your post.

3

I’m sorry that the original poster didn’t reply to this thread from long ago. I’ve found many uses for DTP, and now DTP Office, over the years as the data for one of my research projects have grown to massive proportions. I agree with the original poster that I’d be able to use the program even more brilliantly were there some options for coding within documents, especially of rtf and pdf varieties, though other possibilities might be considered as well. I understand that there might be considerable coding challenges involved in supporting different formats, but I’m not sure why the current tagging system would have to be the engine that made that possible.

In either event, I’ve recommended DT to many colleagues over the years only to have them choose programs like MaxQDA (http://www.maxqda.com), which allow them (albeit less elegantly) to do many things that DTP does and to code documents the way social scientists do. It may be that you’re willing to sacrifice such potential customers in favor of the installed base, but I do see them as a missed opportunity for you.

1 Like
4

As someone who does qualitative research myself, I don’t know if I’d want DTPO to go in this direction. I typically use Atlas.ti for Mac and it suits my needs well. I specifically need thematic coding, quoting and quote extraction, and network links. I also need some of the more sophisticated searching like grep, which is offered by Atlas.ti and other applications.

I think the feature set required to build a reasonably QDA functionality into DTPO would be a rather significant diversion from the core functionalit(y/ies) of DTPO.

What I’d rather see is QDA programs implement a function similar to DT’s AI. I could use AI in Atlas.ti, but I don’t see much need for Atlas.ti in DEVONthink.

That’s my 2c. I understand the desire because some aspects of DTPO seem to hint or tease at the possibility of QDA but I think the last mile of that road would be very hard to do.

Of course if they had infinite resources then, sure, build in QDA to DTPO, and also keep up with the development of all the other bits and bytes. But I can’t help and dwell on the opportunity cost of implementing QDA – what features and enhancements would have to be omitted in order to do that?

That said, I find DTPO to be immensely helpful in other aspects of my qualitative research, especially in content analysis and retrieval. I think all researchers should make use of DTPO if only for keeping track of their notes and academic literature/research documents. But I’d favor a dedicated QDA program for doing QDA. An El Camino is neither a good pick-up truck nor a good coupe. I’d rather not have DTPO go the way of the El Camino.

1 Like
5

When looking at DevonThink, it is important to understand what your goals are and what DT, or any program is going to be capable of and how those capabilities are going to be implemented.

Full disclosure- I am both a Ph.D. Candidate and an active litigator, so I do a lot of data collection and coding, and have definite bias in how to go about it. Sorry in advance for a long post.

When you are talking about qualitative analysis, you are talking about an inherently broad “squishy” concept that is used in many different research methods. Software that works well for discourse analysis may not work well for for analyzing survey responses let alone mixed methods analysis combining qualitative findings with statistical analysis. Video and audio analysis tools are vital for some forms of research, but are extremely complicated and expensive.

Probably the best analogy I can give is of the multi-tools made by Leatherman, Gerber and others. They do a great job at a huge variety of jobs, but I wouldn’t want to rebuild my car or tune my piano with them. For those hobs, you need a specialty tool. Likewise, DT is great for a lot of jobs, but it is not the ideal task for certain specialized jobs.

All of that being said, one of the principal ways that DT can be made more useful for qualitative analysis is by breaking large documents into small ones and tagging them. If I am going through months worth of e-mails, and need to identify specific issues, for example, the best way to do it in DT (that I am aware of, don’t claim to be an expert) is to divide the overall PDF or text file into as close to seperate e-mails as I can and tag each. There are other specific programs that do this specific task better, but can only handle PDFs, or .txt files and can’t do nearly the range of operations that DT can do.

Ultimately, DT is probably best used as a central repository for research. If you are working on a Ph.D., writing a book or otherwise investing years of your life, then investing in specific research software (which your institution may have, or may have discounts for) for specific tasks may be your best option for high-level work.

1 Like
6

I don’t feel like implementing QA tools in DT would be a good idea. Not only they require full coding capabilities, but many also have other very useful features to help transcribing audio or video documents, but also powerful tools for data visualization and analysis — something that I really don’t see in DT without making it become extremely messy. And it would probably still do it less well that specialized softwares.

BUT I agree with others on this thread: I definitely think that for a future main update (DT3?), allowing not only tagging of specific parts of documents but also the AI to work inside documents and between parts of different documents would definitely take DT to another dimension.
Sometimes I feel that it is very frustrating to have the AI suggesting a long pdf article for which I know only a small part will interest me in regards to the initial document. I can’t believe that the only way to achieve this is to cut the pdfs in small chunks by myself, it seems a bit ridiculous.
It would be amazing if instead the AI could directly suggest, giving it’s very best shot, a specific part of the document the it ‘thinks’ is the most relevant. The same way, I would like to be able to select a specific paragraph of a given text (because it contains what I’m specifically interested in, unlike maybe the rest of the document), and give me the most related documents or parts of documents to it. I think that for the people who are really fond of the AI and use it, those capabilities would easily make it a 100 times better.

with hope,
r

1 Like
7

I am interested in using DEVONthink in conjunction with a survey software in the case of qualitative research. Can anybody recommend me such a specific survey software? In particular, I found to my astonishment that in most survey software it is not possible for a respondent to write a longer text as an answer. I’d very much like to have that functionality too, if at all possible.
But any experience of a survey software feeding into DEVONthink would be welcome.
Many thanks,
Adrian

8

I’m not sure exactly which survey software you’ve used, but most survey software I’ve used (Surveymonkey, surveygizmo, qualtrics) all allow open-ended answers.
The utility of DEVONthink in terms of actually analyzing open-ended answers to draw inferences I think is limited — that is not what it is designed to do and cannot possibly compare to the types of coding, quoting, x-references, and analysis that programs like NVIVO and Atlas.ti allow you to do.

1 Like
9

I would find simple QDA features in DT very useful - for example, the ability to tag sections of a PDF from a drop down list of user-defined tags and a feature that would gather the tagged text into a new RTF document with source URLs leading back to the source PDFs, showing the location of the snippet.

10

Yes i’d agree. I have Maxqda but barely use 90% of the analytic features and find its non-native pdf handling exasperating.

11

IMHO, it would be very unlikely.
Nvivo/MaxQDA are specialised apps to treat each individual piece of text (word, phrase, sentence, paragraph) as a basic unit of analysis and that’s why they are slow and laborious for initial setup and ongoing refinement of schemata. DT/concordance operates at document level. For example, tagging is a document property and MacOS based, changing tagging to text-based property means a total change in the back-end. We probably are talking a totally different design architecture here.
See some of the related discussions here: How do you use new DT3 features?

1 Like
12

I’m not a programmer so I can’t say how difficult this would be. But it seems to me that the new annotation function could be adapted to allow snippet level analysis. For example, allow the user to select an annotation and then tag it from the list of existing tags or add a new tag. Then add another feature that allows the user to create a new rtf file that aggregates all the snippets tagged with a specific tag or combination of tags, along with their back links so users can click on the link and get back to that position in the original file. This would allow people to use DT much more productively in their research and writing work flows. I have NVIVO but it’s no good for storing a large number of files, searching, etc. and I don’t need many of the advanced features for my work flow. If DT just incorporated the basic tagging and aggregating function for snippets I think a lot of people would find it useful.

2 Likes
13

I’ve only recently started using DevonThink for file management and absolutely love it. That said, having an ability to highlight and tag within documents, and export highlighted text by tag with reference to the originating file, would be AMAZING and make this really the only tool I would need to use. I believe I could speak for many peers on this point, also. So, just adding a strong +1 to this.

1 Like
14

Check the two links mentioned in this post Accessing annotation command "Insert quote" via keyboard. It may or may not be what u need, but the two scripts both try to break down a document into bits of info and tag them. Just showing that the function is possible in DT. Obviously, these functions are not QDA or NVivo.

1 Like
15

This is helpful, thanks ngan! I think with a bit of noodling I can get the annotation pane script to do what I want. Would be great to have it integrated with existing highlight and annotation features, and a more intuitive way to correlate tags into a single file, but… this is pretty dang useful already.

Really appreciate you taking the time to direct me there.

1 Like
16

I think OP mentions that annotation pane V3 might not work in DT3. But (1) you may want to PM the developer if u have issues, I think he is one of the top script contributors in this forum and he may invest some time in updating the script. (2) User scripts may not get regular update and most of them are developed for personal usage. I only meant to show you the potential of DT but a lot of DT3’s std functions are very powerful and may best serve your purpose I’ll suggest reading their excellent manual and discussion in this forum for treasure hunting.

Cheers

17

I don’t do qualitative analysis. But perhaps this is a basic function/tool that u are looking for?
A suggested basic qualitative tool by combining "summarize highlights" and "Split Document".
The DT’s concordance function may also becomes more relevant when the function is used to analysis a more categorised set of text/snippets, too.

18

I recently had my laptop out of commission midway through a project I had started in MaxQDA so I moved all my remaining files into DTTG. For this project, it has worked surprisingly well - but I’d really love intra-document tagging!

1 Like