Keep in mind, if I toggle that little flag back to database, I can reproduce that exact video I posted before. So this, for me, is highly reproducible. It’s not intermittent or a “perhaps I did it wrong” situation now. There’s definitely a problem with how the files get moved from the Download Manager to the Database that is different from Download Manager to Finder.
Thanks for having me try it. I was sort of being dumb. I thought the URL was something that was unique to DT3, but in fact, it’s standard on the Mac. I’m still used to Windows and unless I use a tool like FDM (free download manager), I lose the URL information because it’s not stored in the file system on Windows like it is on the Mac. So you can see why I was thinking I needed to write directly into the database with DT3, I hope.
My situation is fixed, but somebody else may have a workflow that requires Download Manager to be reliable and write to the database even if they send 1000’s of files. I think that is broken. For me, it’s absolutely the case and I will be paying close attention as I go forward if I switch it off of download to folder…
Just as an aside, I use the Folder option for the Download Manager. When you do this, it segregates the downloaded data by domain, keeping dowloaded items separated from each other. I then index a particular domain directory into a database and work with them as I normally would.
I’m pretty certain that as I was searching for a solution to my problem I saw you mention this before in another thread, but I didn’t understand why that would be useful.
Now I see why and I think I might start this as my practice also since it obviously works well.
I have to learn more about the index (rather than import) and understand if I can benefit from all the other DT3 Goodness like annotations, tags, flags, ratings, etc for the indexed files that are not physically in the DT3 database and see if that meets my goal with this project.
Now I remember why… The site I’m working on is multi-domain.
As soon as I try to do something complicated, I end up with files everywhere. I really have to curate this one by hand.
Might be helpful if DT offered a download to current group option and just let me use the web archive, drop it in a folder, alt-click on each link and have them appear in the same folder without needing to fiddle with a parameters menu.
Download Here Now (and make sure to let me know if you have problems). That’s the button I need. And even better would be something like free-download-manager where you can say “download all visible links here now”. So it’s one-click and done. Knowing what I know now. I may just use their tool in fact. Less brain damage for the DT3 team and it’s their specialty…
You’re right that it should either import the stuff or log a message in case of failure. However, so far I couldn’t reproduce this, that’s why URLs would be great (in case that you should experience this using public URLs).
There’s also an optional script Add Downloads available via the support assistant or the sidebar’s Extra tab which adds only links named “Downloads” to the queue. This could be easily customized to use different names.
I am seeking to download the 15 statements of evidence. What I do as I find the links in the website using the information inspector and then select the items and transfer them by right click to the download manager. This is a very effective tool when having to download items from the website.
I have tried such things as turning off the automatic purge and disabling rules.
There are 15 items of evidence I put into the download manager. Lower screenshot showing mixed downloads on two attempts. The 15 statements of evidence are only partially downloaded. There are no relevant log details.
Then I did a second download to finder and I copied the finder folders into the public docs group in DT3. The count shows 260 and I can see the non-duplicates when I open the folder, so I know they are unique…
The log did show errors with this dataset.
That’s something I didn’t see on my original example.
However, we can see that the 4 errors repeat here. So they are not unique to the database vs finder download… We have 8 errors, one set of 4 for each type of download.
Only thing I found a little weird is that if I set up a comparison of non-duplicates I end up with only 41 instead of 42…
The issue of downloads not working properly to a specific database should be addressed in the next maintenance release. Thanks for your patience and understanding.
