DT Slow with Many RSS Feeds?

DEVONthink Tips
1

Has anyone else noticed a major slow-down when using DT for (many) RSS feeds? I have 1600 feeds in 400 groups (research project). DT is basically unusable, with constantly spinning beachball for just about any operation, click something else, more beachball… Is DT not optimized for RSS feeds? Should I use NNW or another reader, and bring only selected items over as needed? I was hoping to do it all with one tool. I’ve got a newer MBP with 2Gb memory, and this is the only thing running… Thanks

2

I’ve found anything past a few dozen feeds will choke the database and have moved nearly all rss feeds to a dedicated dt2 dbase which thrashes for a while every time its started, but doesn’t slow down the rest of my databases.

I moved away from NNW to try to have less apps doing things that devonthink could handle on its own, especially apps like NNW where I’m not doing anything with the information I find, except finding ways to import it back into devonthink. I figured I’d skip the extra step of NNW and remain in devonthink.

So long as you move rss feeds into their own database (if you have a lot of them), it works ok.

3

Thanks. It will be frustrating if I need to build several DBs just to handle the RSS feeds. Like you I did not want to use multiple apps if it could be helped. NNW is great for reading, but not much else, and one could easily spend significant time writing integration hooks between the apps, to do what DT should already do.

4

How many news do the 1600 feeds contain and how often are they updated (see RSS preferences) and how often do new articles arrive? Could you sample the process using /Applications/Utilities/Activity Monitor? Thanks!

5

Thanks Christian. Here are the stats:

1,800 feeds
65,000 HTML Pages in those feeds
4.5M Words total
RSS feeds updated “Once a day”

Upon starting up, with only Safari running alongside, I get the spinning beachball for quite a while, then stops, click on a menu, beachball starts again, over and over, then eventually one can navigate the app. Activity Monitor shows:

Activity Monitor (2GB MBP with DT and Safari running only)
Free: 15MB
Wired: 457 MB
Active 1.03 GB
Inactive 524MB
Used: 1.99GB
VM Size 143 GB
Swap Used: 242MB

For DEVONthink listing in App Monitor:
Real Mem 184MB
Virtual Mem: 197MB

Interesting thing here is that DEVONthink goes from black to red “application not responding” status several times over the course of 5 minutes in App Monitor, while no user actions are being taken, and goes all the way up to 30% CPU at times. Eventually it settles down and I can use the app. Perhaps this is a great opportunity for me to get a memory upgrade…

6

Of the apps I’ve tested (though none too recently) NNW has always outperformed others at handling a relatively high number of feeds.

7

Any chance you could upload the database (after exporting it via File > Export > Database Archive…) so that I could check this over her? Or could you at least send me (cgrunenberg - at - devon-technologies.com) all the feeds (by selecting all of them and exporting them via File > Export > as Outliner Processor Markup Language)? Thanks!