Hi
I have a 20Gb database that has been open and working for a while. I dont use it often so I closed it. It now wont re-open. Nothing happens. Even if I double click it in Finder it wont open. No error, nothing. My other databases work OK.
What could be the issue please? I did nothing to this database other than close it.
Thanks
Have you rebooted?
Rebooted the computer? I restarted Devonthink. Oh I never thought of the computer, I will try. Hang on… might have to wait till the morning if I cannot back to my mac once rebooted (am home now, and its the office mac).
Managed to login to the computer and restart it… but database still not opening. Earlier today I restored one from Time Machine from the 28th December, but still does not work, maybe I need to go back further but am worried about data loss of course. Is there some repair function or something?
Thinking about it, I use WebDav to transfer to another computer which would be up to date, I could re-sync to a working older Time machine restored database or even start again. I would rather not start again as its 20Gb. Must be a way to fix this one? Just dont know what could have happened to it as I havent used it for a while and hardly do.
I got it back. I deleted the file in Finder, then re-imported the database from the WebDav file I am using for sync.
But it would be good if Devonthink could detect whatever issue is causing this and report the error, and also have some sort of database repair which could find and fix whatever the issue is.
I have not actually purchased Devonthink yet… I have been trialling it (I purchased the iPhone app). So far the system looks like it will do what I need, so would like to purchase it, but I am a bit concerned because it seems to be a little buggy (or maybe just needs more error reporting), I have experienced a few issues, most reported on the forum. So now I have everything setup how I want it, before I purchase, I would like to ask is Devonthink updated often? Are the issues/bugs we find and reported like the above fixed often and updates released? I would be OK purchasing with these issues knowing that they should be addressed soon.
Are you using the Sync beta for important databases? The usual warning still apply - it is beta.
Is DEVONthink updated often? You’d have to define often but we are always at work on it and have a very structured approach to development. Do bear in mind that we don’t do a hundred individual patches (and can only fix things that are actually addressable - as some bugs are not reproducible or may be machine / environment specific) . Our updates come fairly regularly but include many bug fixes. You should check out the release notes some time to see how thorough they are.
I tried finding a page of release notes, but couldnt. Would you have a link?
For often, I dont mean every week, but every 3 to 6 months or something.
I am not use what sync, just one I found on your blog, I assumed it was the latest one. I have set sync up and synced to WebDav before I had this issue of not being able to open the database.
Help menu, ‘Release Notes’. The release notes are also included in the documentation PDF, which can be downloaded here.
Its happened again and I have not changed the database at all.
Can DT not pop up any errors when things like this happen to give us an idea of the issue?
Why is it only happening to my 20Gb database, is that too large a file for DT to handle?
DEVONthink isn’t put off by the number of gigabytes of the file. Much more important measurements are the total number of files and the total number of words in the database (see File > Database Properties). But if you are trying to send 20 GB to the Internet and back, that can be hammering hard at your Internet connection.
As Jim noted, the Sync plugin is a beta, and there’s warning of a risk if you are using it on your working databases. If you are doing so, it’s prudent to keep a sound backup of the database, that is not involved in the sync procedure. (Make sure to manually run Tools > Verify on the backup copy, to confirm that it is error free.)
If your database has become so damaged that DEVONthink can no longer open it, send a message to Support describing the problem, and we will walk you through a procedure to recover a new database from the database, using information from the database’s internal Backups of a previous state before the damage occurred. But in that case, if the damage occurred prior to the oldest internal backup, full recovery would be unlikely. So don’t keep on damaging the database repeatedly.
Thanks for your reply, I have some comments to yours though…
No its no where near the limit.
Have to use it on my working database else Devonthink is no good for me and I should not be using it. When is it planned to move on from beta?
As above in the first threads.
How would I know how to not damage it? As stated I did nothing to the database, but it closed and wont re-open.
Its only happening to one of my 3 databases, the one I hardly ever use.
There have been internal tests of the Sync plugin. This is a public beta release, as that will subject it to a much wider variety of computer configurations, software environments, network environments, etc. As noted in the plugin instructions, if problems are encountered the developers would like to receive reports from users, with information about what happened. That can’t be done on the forum. The developers will probably need a considerable amount of information from users in some cases, perhaps a lot of back and forth communication.
From our internal tests, I’d say that users of beta 3 of the Sync plugin are least likely to run into issues with direct Mac to Mac networking, reasonably likely not to run into issues via a Dropbox connection, and WebDAV (allowing for variables) perhaps the most likely to encounter issues.
You ask how damage can happen, and why the software doesn’t “pop up errors” in progress. The Sync code is complex and can handle a wide variety of variables, discovered in internal testing. But now that it is in the “wild” it is likely that additional variables will be encountered. That’s why it is still beta, and carries warnings about possible problems in use.
A future non-beta release will be made when all serious issues that may be found have been dealt with.
What’s “nowhere near the limit”? Your Internet connection? 20GB across any network is a large task (even on a LAN). Networks are not a constant flow of data, not to mention all the intermediate hops and transit points your data flows through. The larger the file, the longer the transfer, the higher the chance of a hiccup in the transfer affecting the file. Is it better than it was 10 years ago? Absolutely, but it’s still a lot of uncontrolled and unknown paths out there.
Also, you do not need to use the Sync beta on your live database to prove its usefulness. This is why we issue the warnings we do - a beta is still experimental even if in a fairly stable state. It’s advisable and easy enough to create dummy databases to test the process. While it useful and interesting to see what it may be doing with a 20GB database on WebDAV, we’d rather see that be a meaningless 20GB of data, not something important to your business or personal life. Along with Bill, Nate, and the rest of our team I strongly discourage you from using this on a database that is important to you.
(Also note that with the current information, I cannot say that it is the Sync beta causing the issue but the advice remains.)
What's "nowhere near the limit"?
I mean the limit of words/files…
57k unique and 883k words
912 files
So I think this is within the DT limit?
The WebDav is on my LAN so takes about 30 mins to import.
I need a sync system for my files, else I wont be using Devonthink. It is pointless for me without sync, I need my files on various computers. I also need a backup and this is a good way to do it. So I hope it can be released out of Beta soon. I dont see the point in me using DT with some test databases.
I understand BETA and am happy to participate, but I see a couple of issues… 1) I have reported some issues, but there has not been any formal bug submission, DT has not replied stating for more information to try to get to the bottom of it to try to fix it, so it does not seem to me my issues are actually being addressed to be fixed and 2) when there is an issue I cannot see any sort of log file or error that I can also pass to you.
So is there some sort of log file generated I can copy and send to you, and some formal bug submission process? As stated I am happy to report bugs if it means you will address and fix them sooner and get us a working system system. I think it about time you have a working sync system.
Thanks
Laurence
As a suggestion, read the ‘About the Sync Plugin Beta’ document that is included with the sync beta download. In addition to containing the information that may help you avoid problems when syncing with the beta, it includes a section on the formal bug submission process and the log files generated in the Console.
This is not backup, it is Sync. This may seem like semantics but it’s not the same thing. (This is especially true of cloud services like DropBox.) You should be observing an appropriate backup routine in addition to whatever syncing you want to use.
The point is to be able to test without endangering your live data. It’s not so much a “bug” you’re reporting (though it may turn out to be a bug) but behaviors. Would you like to keep damaging this 20GB database while you determine the steps to reproduce the damaging behavior? We have not determined Sync is damaging your database so it’s not actually a bug yet but we also have no idea how to reproduce the behavior. We don’t know about your setup or environment and the actual steps you could use to make this happen. This would all be very helpful information but again, I don’t think you want to keep damaging your live database to figure it out - and neither do we. Hence the strong suggestion to use test databases, not live ones.
A reminder from the Sync README…
Also, as noted by Greg, notes on how to report a problem and the information useful to the process are included on page 7 of this README that was included in the ZIP that contained the Sync plugin.
I am happy for my “backup” to be defined as another copy on another computer. I am not bothered about previous versions because both my computers are Time Machined, so I have both previous version on both machines. But if my house blows up, or office falls down, I have a backup on my other machine. So sync for me works as a good backup, I dont need a another (non Time Machine) backup AND a separate sync. Although I may do this anyway I have some spare USB drives.
I honestly cant understand why I would use a test database and hope I repeat the error on that. It does not occur on my other two databases, why do we think using a test database of not live data will repeat the error. So it may be my live data is when the error occurs. So I would rather you just deal with this error on this live database instead of saying do not use sync.
As I understand it, I can still get my files from within the DT database file, just lose organisation? So if thats worse case so be it. I have 3 copies of this database now, could my data be corrupted beyond repair and lose all data on all 3 locations? If there is a chance then maybe DT in general is not ready for release if it can corrupt data beyond repair. If it wont, then I should be OK with 3 copies from which I can extract the actual files. Thats my opinion.
I will checkout the bug reporting process and logs at some point then, because I understand bugs will exist, so am OK to help report issues if it means they will be fixed.
I have done a lot of research with DT and other similar software and in my opinion DT is the best for this, so I really want to use it and am using it, I have made the switch I cant go back. So am very keen to help you improve this software if my bug reports help.
A good backup is one that captures the database in an error-free state and with all data and metadata available. It should be stored on an external drive or storage medium for access in case of a hard disk crash, a lost or stolen computer, database damage, etc.
The Sync procedures do not constitute a backup process, although copies of the database are available on multiple devices. In use, Sync is modifying one or more copies of the database. Sync can be wonderfully useful in helping keep databases “in sync” on multiple computers, transferring changes made such as content additions or edits between my working Macs. But Sync doesn’t protect me from screwing up a database by doing something dumb, then transferring the screwup to all my other copies of the database. That’s when I need a true backup of the database that’s still in good condition.
I use Time Machine. But a weakness of Time Machine is that it will make backups without checking database integrity. It will cheerfully backup a damaged database. If I understand that, I can work around that kind of weakness. I can manually run Tools > Verify & Repair in DEVONthink to verify that the database is error-free, and make the next Time Machine backup more trustworthy. (In normal work with my databases I haven’t encountered any errors for a very long time; but I’ve made it a habit to run Verify & Repair occasionally.) Another weakness is that, because a DEVONthink database is dynamic, there could be database changes happening in memory that hadn’t been saved to disk at the time of a Time Machine backup. Again, I can work around that issue by closing the database before the Time Machine backup. So, the most trustworthy Time Machine backup will be one that was made with the database closed, following the Verify & Repair procedure. (In actual practice, having checked a number of Time Machine backups made with open databases, I’ve never encountered a problem; but in principle I would prefer a backup made while a database was closed.)
My “ultimate” backups in trustworthiness are database archives (File > Export > Database Archive). This procedure will first check database integrity, and stop and issue a message if a problem is found, allowing the user to take corrective measures such as Restore Backup or Rebuild Database. A database archive is the smallest possible complete copy of a database, compressed as a ZIP file. The compression step does take time for a large database. Periodically I’ll update database archives that are stored on a portable drive that is kept in a safety deposit box at my bank. Thus, even if all my computer equipment were stolen or lost in a fire, I haven’t lost everything. I prefer to keep full control of these backups rather than store them in the cloud. As my collection of files is very large, I can restore them more quickly by picking them up at the bank rather than downloading them from the cloud.
Thank you. I will most likely introduce some sort of backup also. I would refer to it as secondary, with the “sync” being my primary because my second computer will have all my documents, and call upon the secondary backup if the sync method introduced issues.
But I am not sure any can be error free state when backed up using any backup process, unless DT can find 100% of errors before backing up.
I just did Tools -> Verify on my problematic DB and it stated no errors found (maybe the issue has gone), so if it happens again then the error would have been backed up too.
If a backup process can guarantee 100% error free then of course that would be the best backup solution. If it cannot then surely it would not matter what backup you use because any of them could contain the error. I am not an expert on backup technology though, but this just seems a logical explanation.
Hi
I am still having issues with this. I just cannot open/import a database.
I deleted the database I could not open and decided to re-import. It looks likes its importing but at the end never asks where to put it. The import pop up just goes, and then nothing happens. Its not imported.
Sometimes when I try to import it does not even import everything but the import pop up just disappears right away and nothing happens.
I have tried importing from a Webdav sync db AND from a local computer.
Note: the database on the local/Macbook computer works fine! This syncs to the Webdav fine. From the Macbook there does not seem to be any issue. It just wont import into the iMac.
I have even tried to rename it on the Macbook, and resync. I can see the new name in the import on the iMac, but same issues, wont import.
I have sync version beta 1.0pb2, is there a more recent version? If so where?
Thanks