At first I wasn’t sure if it was a bug or not, but according to this thread, images in RTF documents aren’t scaled to fit the screen, as DEVONthink uses Apple’s code (and TextEdit doesn’t seem to scale images either).
I usually use a text editor called Bean, which does scale images, and it would be nice to see this in DEVONthink as well…
The developers are using plain and rich text code in OS X and prefer expending development resources on DEVONthink itself.
You can “fit” images to a view pane by using the magnification commands in the View menu (there are keyboard shortcuts).
If you wish, you can make Bean the “parent” application for rich text documents (in the Info panel, in the Finder). You can then open a rich text document under Bean by selecting it and pressing Shift-Command-O.
Since version 2.5.2 (and therefore for more than one year) DEVONthink doesn’t scale images any longer due to user requests. We might add a hidden preference to future releases though.