The problems you mention are also problems I have faced. There is no easy solution. The way I look at it is this: the University Library here where I live (Cambridge) is huge and has works on every field you could imagine. But I can still find the books that I want when I look in the catalogue. And I cannot know in advance that a book or article on psychology will not be relevant at some point to my occasional studies in history. So I have a database called “Library” in which I keep material that might be used in my research and writing. This contains an indexed Attachments folder which is controlled by DEVONthink. In truth, because I work in psychology mainly nowadays, almost anything I find might be a source work and need to be cited, so much material goes into Bookends.
There are advantages and disadvantages in having such a Library database. One advantage is that you never have to remember which database you put something in, because they are all in the Library. Moreover, you never have to decide which one to put them in because there is one main place to put them.
It is true that I have other databases, but they are for matters that are not work-related or academic. And if I want to work on a small project, I can always create a database specifically for that, and collect in it material that particularly relates to that subject. In that case I can index individual pdfs from the Attachments folder if I need to. That gives me a focused environment to work in for that project, but I can always go to the “Library” if I need to. When the project is finished, I can archive that database if I won’t need it again.
Ultimately, however, you have to find a method that works for you.
PS: you might find it valuable to have a look at this article – https://zettelkasten.de/posts/collectors-fallacy/