I’ve been importing large tables of data into DT from files converted into CSV by a script. The data format in the original file is two streams of data, each consisting of 16 segments or windows. I represent this in DT by creating a group for each stream that contains 16 sheets, each of which contains 512 rows (2 columns of numeric data).
To avoid generating a temporary file, I build the sheets as follows (pseudocode):
for each stream {
g = new group
for each segment {
s = new sheet in g
for each row {
new form (cells:column_list} in s
}
}
}
This is, needless to say, pretty slow. I’m creating 16,384 forms for each record. The entire import takes about 6 minutes (wallclock).
Is there a faster way to do this (aside from importing the data as a CSV file), perhaps by specifying the data of the sheet on creation?
–Eric
PS: Just a quick note, doing one of these imports in the root directory locks up DT pretty good.