Slow Response of DT3 Webserver vs DT3 Desktop

I imported data into a new database today with size as below.

I can search for a given group name or document name quickly either on DT3 Desktop or DT3 Web.

However, when I want to browse through the database i.e. to open a group and view the documents, it responds immediately with DT3 Desktop but is quite notably slow on DT3 webserver, almost to the point of being unusable on the webserver.

Is there any way to improve the performance on the web?

A native application is of course way faster than a JavaScript solution for the web.

What kind of connection do you use and which browser/OS?

I am using Catalina 10.15.5

I am using Chrome; I tried it on Safari just now and performance is worse, literally unusable there.

I have a fiberoptic Gigabit internet connection with hardwired Ethernet; actual speed is about 800 Mbps down and 950 Mbps up. But right now I am simply trying to access the web server on the same local computer running the webserver; I have not even tried yet seeing how it works from an external computer.

My other databases work really well over the webserver - it really seems that for some reason the webserver performance hits a point where is cannot scale for some reason.

This database contains a huge number of groups, the average group contains less than 2 documents. Are your other databases similar?

Yes, this database is different in that respect. It contains medical records of a large number of patients (each in a separate group), each of whom has about a half dozen documents spread among a 2 or 3 subgroups.

I thought about it a bit and considered maybe 21,635 is too many groups to resopnd quickly. So I tried replicating those starting with A or B. It still does not speed up that much.

It may be somewhat atypical in the number of groups but I don’t think it’s that unusual. DT3 desktop handles it with no issues at all. I am thinking there must be some aspect of the algorithm or memory management used on the web that slows it down. Interestingly the search for a given patient remains very fast on the web; the slow speed comes instead when opening the groups or subgroups.

image

The search and all other database operations are performed by the backend (server) and are therefore basically as fast as on the desktop. But the frontend running in your browser provides the user interface/experience (navigation, viewing, editing etc.)

And it’s of course a basic limitation of web apps compared to native apps running on the desktop that speed, memory usage & possibilities are still very limited.

I compared the performance of several of my databases on DT3 webserver.

At least based on testing with a few databases, I do not think the problem is a limitation in database size or speed or memory; I think it may be the algorithm used to open subgroups, especially those that go beyond one level.

If I have a database with 20,000 groups, it is actually quite fast on the webserver to list those 20,000 groups. But if I select to view one of those groups that has 3 subgroups, it takes an extremely long time to view those 3 subgroups. And if one of those 3 subgroups in turn has another layer or subgroups, it takes longer still to list those lowest level subgroups.

So why would it quickly show 20,000 groups but then take forever to expand one of those groups to show just a few nested subgroups inside?

The web application will pre-fetch the content of the root level of the database whilst it is loading. When you open a group it will request the contents of that group, i.e. both subgroups and records. The navigation tree will only expand once it has received and parsed the records.

For the groups that you say are slow to open, how long is it taking to open the group in the navigation tree, and how many records does it contain?

There does not seem to be any relationship between the number of records and how long it takes to open - sometimes very small groups take the longest to open.

But I just realized something - the performance issue is basically limited to the left pane.

If I set the webviewer to “Widescreen” mode with 3 panes, expanding the navigation tree on the first/left level to show 21,000 groups takes under a second - certainly reasonable for a large web database. If I then stay in that left pane and try to further expand the groups (most of which have under a dozen records and none of which have more than 50 records) it can take between 1 second and 5 seconds for that expansion to happen.

But the interesting part is that if I do not expand the navigation tree in the left pane beyond 1 level and instead click on a group name so that the rest of the tree appears in the middle pane, then the identical tree expansion which takes up to 5 seconds in the left pane occurs instantaneously in the middle pane. In fact, even if I first expand the middle pane and thus the tree presumably is cached somewhere, again expanding that identical part of the navigation tree in the left pane remains delayed.

** Another user interface comment - When traversing the navigation tree in the left pane, it only shows subgroups but never shows documents. But in widescreen mode the middle pane does show both subgroups and documents. This is important because the caret showing that there is content in a group remains present in the left pane even if there are only documents and no further subgroups. So the result is that when trying to traverse the tree in the left pane, if you click on a caret you have to wait quite a bit of time to be sure if there is no response because there are no further groups or if it is just waiting to retrieve data. It would be much nicer if the document list were included in the left pane.

If you are following all this, then the logical solution would be to just click on the database name on the left pane so the entire navigation tree becomes available in the middle pane. But then it starts slowing down again to expand subgroups, just as normally happens in the left pane. So it appears that the bottom line reason for the delay is that when expanding the navigation tree, the web app is re-populating the entire navigation tree in the pane for each expansion rather than simply retrieving the additional data needed to open one more subgroup.

The web application provides the same functionality that you will find in the main DEVONthink application for standard and widescreen views.

That is not the case, only the data for the newly loaded group is added to the tree. I will look into what is causing the slow expansion of the tree items.

Not exactly

This is from the desktop - it is immediately clear upon expanding “TEST A” that there are two subgroups:

image

But on the web the subgroup expansion does not always occur in the left pane - this is the same database - Test A never expands to show its underlying subgroups so that adds to the delays because one wonders if the expansion is complete.

image

Thank you - much appreciated