I am curious to know how others use some of these features!
the new concordance interface
I know what they are for, but practical examples would be great (I think) for all of us.
About the new concordance, one thing I thought of suggesting to @cgrunenberg is allowing similar words to be grouped together so that they would appear as a single thematic cluster. I don’t know how feasible this is though.
Disclaimer: my comment/discussion can be incredibly naive due to my lack of knowledge in concordance and the typical application of word cloud/concept schemata, and qualitative method such as discourse analysis.
Serious utilisation of word<->concept<-> schemata probably falling more into the scope of more specialised software such as Nvivo. Nvivo is kind of de facto for qualitative and mixed-method researchers (I think?), but it is extremely “slow” and requires laborious initial set-up and ongoing fine-tuning to make the connection works (I tried testing that app a few years ago). I guess there is no easy way when it comes to connecting words and concepts.
The core mechanism of concordance may be essential for some DT’s core function (classification?), but I rarely see concordance of DT being discussed in any forum/blog. Very personal opinion: for concordance reaching to the point of being approachable (ease of use and lots of examples in practical application) and usable (flexibility in customisation), a significant amount of further development and reference to the design of other niched apps may be required. The problem is, perhaps only very few DT users will ever need that sort of ability “directly”. IMHO, concordance may begin as one of the core competence in DT’s blueprint, but now it seems more like a mechanism that is working hard behind the scene. Just to be clear, DT is an amazing, incredible, and unique app in my eyes! Smart rules and smart groups are already delivering everything I need!
Thank you for sharing this! I created not one, but two smart rules based on this.
Speaking of which, @BLUEFROG there appears to be a small error in the date detection on DT3. Dates outside the US usually are written DD/MM/YY and not MM/DD/YY. When the day is <12 then it understands it correctly, when it is >12, then it will wrongly misidentify it as being the month.
I agree with you and yes, Nvivo, MaxQDA, Atlas.ti are all softwares for quantitative and qualitative data analysis. My suggestion would bring DT3 closer to them, but I can see why this would be difficult: these are all big companies that charge thousands of dollars for their software.
As a matter of fact, I have a temporary student license for MaxQDA and while I do find it useful from time to time, DT3 is much closer to what I really need. Most of the time I use it in a similar way to what a windows software called Connected Text does very well, perhaps you heard of it. Still DT3 is obviously richer in features and more open-ended than Connected Text and from time to time I do take advantage of that.
I read some/most of the threads about concordance in this forum. I feel that I am totally wrong to compare Nvivo/MaxQDA and concordance. Nvivo/MaxQDA are specialised apps to treat each individual piece of text ( word, phrase, sentence, paragraph) as a basic unit of analysis and that’s why they are slow, concordance operates at document level.
My interpretation: The salient goal for concordance is to identify the word-usage of similar clusters of documents in DT. Very broadly speaking, the most crucial element in concordance is the differential weightings of words in word-cloud, not frequency rank. A cluster of similar documents is differentiated by how different is its top-ranked words when compared to ALL other clusters. This means that the primary/sole purpose of concordance is implementing “classify and see also” and the efficacy is limited by:
(1) How good is the quality of text/OCRed files? If many pdf files have bad quality OCR (words sticking together, incomplete recognition, many random mixtures of symbol+word in rtf and mark-up files etc.), the differential weights of the top-ranked words will be meaningless.
(2) The quality and the characteristics of existing groups set by users. If users can (i) put “really” similar documents in each group, and (ii) the topic/subject of groups are significantly different from each other, and (iii) exclude many irrelevant groups from classifying, then concordance will be able to learn and identify the unique pattern in each group and do a good job.
This also means that:
(1) We can’t expect DT/concordance to do anything similar/close to the functionality of Nvivo/MaxQDA jobs other than “wishing” DT may have the resources to extend their core competence by improving the functionality of word-cloud . It is because the current form and properties of word-cloud in DT is already an efficient and sufficient mean for “classify and see also”. EDITED: It may means that (i) concordance becomes a back-end engine and word-cloud/analysis is separated into a function that allows a higher level of user-customisation. (ii) concordance becomes customisable and allow users to have their own ways of “classify and see also”.
(2) That’s also why it is much more challenging to perform a meaningful auto-grouping: how accurate could DT cluster a bunch of new items into new groups without a preexisting reference of uniqueness? Perhaps all DT can do is to group the obvious into existing groups, and compare the rest with plain-vallina frequency-ranked word cloud and try its best to cluster them. If I am a very demanding developer and know the limitation of the current methodology, I probably won’t be happy with this incomplete solution and would rather not to include the function in DT3 (pure speculation + imagination). Kind of like what a good chef will do. EDITED: otherwise, if users just let the auto-created/assigned groups staying there, there will be negative learning coz those groups will become increasingly “garbaged”.
(3) Smart rules/groups + tagging are better answers to target grouping according to the design philosophy of DT (another pure speculation + imagination).