Sheets and Records in DTOP2

In my database with DTOP1 I worked with sheets of about 10,000 records. I viewed all the records in a three panel view and edited the records in a singular window. When I transferred the databases and the sheets in DTPO2 I can’t display the records in a three panel view. The records remain in rows view.
How can I resolve the problem?

Isidoro

DEVONthink Pro Office 2.0 does not have a record view anymore. It may return eventually at a later point in time.

But Do You foresee it?

C’mon, cut him some slack about it. :slight_smile:

After the v2 brouhaha I hope Dtech is careful about making any statements that might be considered promises, potentially leading to false expectations and disappointment if unfulfilled.

Christian told me to tell you: Thank you for the suggestion. We will consider it for a future release.

It really wasn’t a suggestion! I really don’t understand why this function has been aborted on version 2.

They had about a billion suggestions and implemented a great many of them. I’m sure (keep in mind I’m not affiliated, etc) that it’ll reappear soon, since a lot of people are asking for it. I’m also sure (or at least hoping) that it’ll appear in a greatly improved form, perhaps utilizing HTML formatting instead of the rather drab solution in v1.

I agree completely. This is a feature that I had become quite dependent upon.

The only thing I can figure is that Sheets were handled like folders before, and therefore allowed the three-panel view that you and I grew accustomed to; but now, for whatever reason, they aren’t being handled as folders any longer. (That’s the only thing that makes sense to me, though I know nothing of what is under the hood here.)

It seems to me that, if my hunch is right, an option could be installed to toggle this off and on-- maybe a preference checkbox to “View Sheets as Folders” or something.

If it is a feature that is entirely abandoned, that renders DevonThink significantly less useful to my daily workflow. I would seriously consider returning to v 1.x to regain this feature-- or would abandon using DT for these records altogether, which would result in less use of DT for other things as well.

Yep. Each sheet is a tsv (tab-separated value) document now instead of a virtual collection of virtual rows. So things are different, and there’d probably be more work than we might assume to do what you’re talking about.

What I hope will happen in DT 2.X is that they’ll shift to an XML representation. XML allows typed fields, arrays, key-value pairs, and crap like that while still being human-readable and “open” and easily interpreted by programs. Since AppleScript in Leopard handles XML much better than in previous versions of OS X, it should be a relatively minor problem for anyone with a substantial number of DEVONthink sheets to transform them into a more compatible format for use by other programs.

In my dream scenario, each DEVONthink sheet would be a bundle containing a single XML file describing the layout of the sheet and (potentially many) XML files each representing a record. It’s possible to do this in a single large XML sheet, but it might slow down some operations requiring repeated reads/writes on records (for instance AppleScripts) since XML is a rather “wordy” format and each file naturally has to be saved in its entirety every time a write is saved, and then read again for the next record’s read. It might not make any noticeable or important difference.

The sheet format (a bundle for each file) could also be a way of storing extensible metadata with files. Instead of kalisphoenix.rtf, you might have kalisphoenix.rtf.dtDocBundle or something similarly named, and each bundle would contain the file, its associated metadata in XML format, and any other information that needed to be retained reliably. This could be utilized by DEVONthink when it performs a [b]Rebuild Database...[/b], utilized for transitioning to another application, for creating a new database from a selection of files, or for any other situation where you want to preserve all of that information independent from DEVONthink's database and backup databases.

A hook into the XML metadata-dumping process (even if only through AppleScript) could easily provide other applications with ways of browsing or visualizing the DEVONthink database. For instance, TouchGraph, which would answer some requests by DEVONtechnologies forum goers of having a mapping system similar to that of DEVONagent built into DEVONthink.

If that is the case, then I would actually be far better off simply creating a new simple text document for what would be each record, if there were such things as records in sheets anymore. At least then I would be able to see the whole set of data in a readable format-- and I would actually gain the capability to cut and paste more than one “field” at a time.

In fact, I would go so far as to say: if that is the case, then Sheets are useless to me now-- and I have difficulty imagining a circumstance in which they would be useful to anyone else.

It doesn’t make any sense to me that, by virtue of a major version “upgrade”, you actually lose major features and functions.

It’s my hope that the single-record view will be re-instituted soon, preferably (at least in my mind) with a way of creating forms in HTML so that we can define how the records are displayed.

I agree… I just hope that it’s a temporary loss in functionality, and that what will replace the old sheets will be very, very useful and powerful.

I’m with you in that I want to see the records view back, but please don’t assume that everyone uses DT in the same way as you. I use sheets to transcribe census records and only switch to records view on occasion.

Karen, I haven’t assumed that-- however, I maintain what I originally said: “I have difficulty imagining a circumstance” in which Sheets, in their current form, present any benefit.

So, feed my imagination: how do Sheets benefit you as they presently function? Will you describe your usage, so that I might better understand? And specifically, what advantage does this format present that wouldn’t be as useful in a simple rich-text document?

Ed, I think Karen was commenting that Sheets remain useful for her to transcribe census records in DT Pro 2, as a counterexample to precisely what you had said in a previous post:

But she also supports a reintroduction of the Records view. :slight_smile:

Yes, I read that Bill-- which is precisely why I responded the way I did.

I understand that she uses it to transcribe census records. If you’ve followed this thread at all, you’ve noticed, perhaps, that I have a lot of data in Sheets too-- as do others. Whether Sheets contain data isn’t in question.

My question is, I thought, a very simple one: of what benefit is putting data in Sheets-- in their current form-- over and above simply using text or rich-text documents?

Perhaps I need to take my question a bit further, since I’ve obviously asked it in a dull manner. Let me give an attempt.

In my view, the primary advantage of Sheets in DT 1.x is that they offered something of a hybrid database model. Whereas DevonThink is an open, freeform database of the highest order, there are clearly occasions when a more structured database is useful. Sheets allowed DT users to build structured databases within the freeform context, and offered the usefulness of DT’s powerful search and AI functions alongside it, giving a nice hybrid of the structure in microcosm with freeform, AI, etc. at a larger level.

Without Sheets, I could imagine a couple of ways a user might manage similar data in v 1.x. One could, for example, create a single document (text or rich-text) with the first line being something equivalent to labels of columns, and each subsequent line being the equivalent to a record. The data could be separated by tabs, maintaining something close to neatness and keeping export into spreadsheets relatively easy. (This, incidentally, would be essentially identical to Sheets as they currently exist in v 2.0b.)

On the other hand, if that presented too much data in a line than is easy to deal with, or if it was difficult to view all of the data for reference, etc., one could create a FOLDER in DT, containing a set of documents that each would be the equivalent of a record, containing the data in whatever format the user preferred. You could even go as far as to set up a template (maybe with each “field” already labeled and separated by tabs or hard-returns) that you would duplicate and rename, etc. This option would be a bit less automated, but would functionally approximate Sheets in three-panel, Record view in DT 1.x. (BTW, this is basically what I did before Sheets were introduced, but I was much more satisfied with the way Sheets handled the data.)

So, here’s my question, restated: is there some value or benefit to Sheets that isn’t in some way contained in these ideas? From my understanding, Sheets don’t offer any computational value (in the way that spreadsheets do, for example). Is there something I’m missing? Why put data in Sheets if you don’t rely on the individual Record view?

I’m not being sarcastic-- I really don’t know.

I like sheets as implemented in DT 2. One thing that’s useful to me compared with a rich text document is that you can easily sort and reorder data using sheets. So I can click on different columns and reorder the records. Maybe it’s possible to do that in a rich text file, but if it is, I don’t know how to do it.

I guess I wouldn’t mind seeing at some point a single record view (that looked nice in v1.x, but honestly I never found a use for it). In general, my experience is that I’ve never used that sort of interface to enter data (e.g., in filemaker). I guess I personally don’t see this as a loss of functionality, because it’s still possible to enter data, it’s just that instead of seeing 1 record at a time, you see a table, with 1 record per row.

I like the way I can export a sheet and then import into excel.

Sorry Ed, I misunderstood your original post. When I enter data into a record in the “sheets view” (for lack of a better term), I’m comparing data from the record I’m entering to the records previously entered into that sheet. I start with a template sheet that I’ve created for each census year, duplicate it and start entering the household members. So, if I’m entering a record for Elizabeth Adams in the 1850 Census, I’m also looking at the records I previously entered for her father Elisha, mother Elizabeth, and her older brother Adam. Especially helpful if a family has a lot of kids where it’s easy to lose your place and skip kids or enter them twice. If I were to just enter them into a record view, I wouldn’t be viewing the information for the other family memebers. If want to include this data in a document, I copy/paste it into TextSoap, replace the tabs with spaces, copy/paste into my document.

I realize that I could also create a rich text document and do something similar with a table, but frankly, I have a lot of problems with tables holding their formatting in Mac OSX. Also, tables are no good if you switch to plain text.

OK, so that probably did more to dampen your imagination than feed it, but that’s how I use them!

Mmm. You can do it with a tab-delimited plain text, at least (and since Sheets don’t work with rich text, I dunno why anyone mentioned rich text). It’d be a very simple AppleScript based around this command:


cat $filename | sort -t \t [-r] [-n] -k #

Even have numeric sorting, date sorting, and some other goodies that (IIRC) aren’t present in the current incarnation of sheets. (I’m actually a bit surprised to see that this doesn’t already exist as a Service…)

Might I ask how many columns you’re working with? Are you doing this just for one family, or for multiple families?

It’s not that I can’t imagine how sheets would be useful to you… it’s that I can’t imagine how they couldn’t be more useful. Filtering records in a sheet, for instance (as mentioned elsewhere, I’d like to see Sheets handled as key-value pairs, so you could do a search within a sheet for “BirthYear=1850” or “BirthPlace=‘Martin, KY’” or “MothersSurname=Adams” or whatever). Or adding typed fields so that you could accurately sort by birthdate whether it’s “3/15/1850” or “March 15, 1850.”

One note is that I think many people miss that the single record view isn’t useful just for data entry, but for viewing a single record. It’s more useful in applications like Filemaker, where you can store information like images, PDFs, videos, even related records within a single field. Imagine a sheet with fifty or so records, with a picture (gotta love those cheery sepia-toned pictures of old bastards easily found on the internet) or a map to their home or gravesite or birthplace or whatever. Or (as I mentioned in a post in the beta forum) making a sort of low-rent version of Google Maps with an HTML form and some small ability to program metadata.

Right now, sheets are plain text documents with tabs between columns. That’s it. Okay, and you can sort the columns alphabetically or reverse alphabetically.

Now, Ed Eubanks:

You’re precisely right, IMHO. Sheets should, as far as I am concerned, be a way of introducing structured database features into DEVONthink. Basic databases aren’t complicated. Relational databases are a little more complicated, but we’re not all that worried about performance… we’re not running this app on 486’s, and we’re not serving Wordpress or Digg or Reddit to thousands of people every second. You could implement a relational database in AppleScript, fer chrissake. You could probably even implement most of the SQL standard in AppleScript. It’d be completely retarded, which is why I haven’t done it, but it’s possible.

This isn’t about making DEVONthink everything to everyone… it’s about rounding out a feature that theoretically already exists, and making the application more solid and feature-rich. Big ROI.

roberthoodphd said:

Yes, that is a good feature-- and I had omitted that it is preserved in DT 2.0. Thanks for mentioning it.

KP said:

Thanks Karen-- that’s helpful to know. I can see how, in that sense, Sheets would remain useful to you and the way you use them.

kalisphoenix said:

Yes! Yes-- exactly. This is one of the HUGE things that I miss already about Sheets.

The way I use them, Record view is quite useful for entering data, though I, like Karen, often do comparisons of data (more on this in a moment) and find Table view helpful for this, if modestly so. But when it comes to retrieving data from Sheets, Record view is essential to my workflow.

It occurred to me that it might be helpful to give an example of how I use Sheets, so that others could understand why I so miss Record view.

I’m a pastor, and I use DT to manage, among other things, the weekly worship services that we hold. So I have a Sheet with 24 fields/columns, each containing a different piece of a worship service (opening hymn, opening prayer, benediction, etc.). Some of these contain multiple paragraphs of data, and viewing them in Table view means that either most entries are truncated or I’m endlessly scrolling to the right.

I use each record in four different ways: as I’m planning a given worship service, I spend a lot of time with that record as I identify what the theme of the service will be, which readings of Scripture are appropriate, what hymns to sing, etc. (I usually begin this well in advance, even more than a month.)

Next, I reference each Record as I’m reporting to our Administrative Assistant what information should be printed in the order of worship-- she’ll print a good portion (maybe 35%) of the content, and I reference the record in sending this to her.

Also, I create a “script” or guide for my personal use when leading worship, that has all of the information in the worship folder, plus a good bit of the content filled in as well-- I use some prayers, readings, etc. that are already written. I rely on the Record heavily as I’m preparing these pages.

Finally, as I’m preparing any given week’s worship service, I’m compare that week to others, referencing them for how frequently I’ve used certain hymns, prayers, etc. I also use older services when constructing new ones, copying a prayer from this one and a confession of faith from that one, avoiding re-typing as much as possible.

All four of these COULD be accomplished in Table view-- and you might even argue that the last is best accomplished in Table view. The problem is what I’ve already mentioned: the fields in each column are either so wide as to be obtuse, or they significantly truncate the contents. If DT 2.0 offered text-wrapping within fields, that would even be a move in the direction of improving this-- but it still wouldn’t replace Record view as far superior, in my mind, for these purposes.

I’m at over 400 sheets (one sheet per family per census year). Number of columns vary from 7-24 (census data collected varies by year).

Kalisphoenix, I like your suggestions, but I have a feeling they would be better implemented in a dedicated genealogy program or Filemaker. But who knows? A programmer, I’m not (unless you count Fortran 77, and with that I used to rock!) My dream would be to see sheets be able to handle rich text instead of just plain, so that I could link an entry to other files in the database. I would just be happy to have records view back and I like that sheets are no longer stored within the database file itself, but in .tsv files. Big improvement. I’ve noticed that opening DTPO2 is much faster.

Scrolling=bad. I see why you need records view.

I also noticed that the keyboard shortcut to add a record is gone. :cry: I would like to see that come back!

Cheers!