Just a quick question. I read about bit rot. And that it is easy to create a single disk image from a directory that, for example, contains DEVONthink databases. Is this a good idea periodically, in addition to other means of backup? (On the basis that disk images are more capable of recovery in the event of bit rot).
Me, I just do multiple backups using the 3-2-1 strategy (see multitude of articles on Internet for thatI. I also test recovery every so often so that I can ârememberâ how to do âem and test (best I can) the backups actually exist and are restorable. I also have a weekly automated script that archives to zip all my DEVONthink database to a local folder that then in turn those zip files get backed up.
I donât worry about bit rot. Maybe my bad, but with 3-2-1, if a needed backup not restored as I want, I always have a probable good one elsewhere. And I am guessing that a disk image will copy any rotten parts of the disk, if they exist and not fixed by the disk software, anyway. No?
3 Likes
Iâd first like to give some perspective on how likely a âbit rotâ (also often called âbit flipâ) incident would even be.
I worked for a while in a scientific datacenter that operated a custom storage software that watched over many Petabytes of disk and tape storage. That system had a random check to see whether files still matched their checksums. And actually, we saw a handful of such events over the course of a year â the stored files seemed to have changed in subtle ways so that their checksum did no longer match the one that was stored along with them on file creation. So yes, things like that do happen, and they do happen on journaling filesystems and RAID arrays â those donât really protect against random bit flips. However, imagine our datacenter as having around 40 rows with 36 racks each, and each rack housing around 300 disk drivesâŚ
So, on a typical Mac-based home user setup with perhaps three drives for your computer and external backups, this is not realistically going to be an issue. There are many more common causes for data corruption, and you should be fine with a classical robust backup scheme.
4 Likes
I was more thinking about degradation after backing up. I think a degraded image will open whereas a degraded zip wonât. Could be wrong, and I am sure I am over-simplifying. It is the data that is of an archive nature that is presumably most at risk of bit rot.
I guess, however, that your datacentre always has power. My understanding is that someone like myself who puts an ssd in a drawer for two years is much more at risk because an unpowered ssd slowly corrupts. And my further thinking was that since a disk image can cope with mild degradation better than, for example, zip files, it might make sense occasionally to save as a disk image. I also believe that daily backup archives are more forgiving of bit rot, but I have so many databases that looping through them with a script breaks. So, for the lazy person like me, I am just wondering whether the creation of a disk image (easily achieved) might make some sense. But I do agree that in terms of levels of risk, bit rot probably comes very low (even though it sounds unpleasant!). I donât think bit rot is a problem in the cloud, so that also is an answer (provided you renew your sub etc).
3 Likes
Why would that be? And what is âmild degradationâ?
I read this (which might not be true, or might need nuancingâŚ)
Why a Disk Image is often âBetterâ
If you use an uncompressed disk image (like a raw .img or a standard .iso), it is generally more resilient for two reasons:
Lack of Interdependency: In a ZIP file, files are often compressed. Because compression relies on patterns, a single bit error can âexplode,â making everything after it in that stream unreadable. In a raw disk image, a bit rot error usually only affects the exact byte where the rot occurred.
Built-in Redundancy (ISO 9660): Standard ISO files were designed for optical media (CDs/DVDs), which are prone to scratches. They include significant overhead for Error Correction Codes (ECC). This means the image can often âself-healâ small errors that would permanently break a ZIP file.