Only 10 stupid backup strategies

DEVONthink Tips
1

In some sense this is somewhat off-topic, but I want to remark on the sticky forum item
[url]11 Stupid Backup Strategies] on the top of this list, which discusses the article “11 Stupid Backup Strategies”. That thread is unfortunately locked, so we have to take the wisdom in that article for granted.

Generally, I like the article, and most things are correct, but I completely disagree with point 4 (and Bill already makes similar points):

I agree that this is not a good a idea to do this only sporadically. But if this is done with discipline, it is in my view the single most reliable and safest method there is. Telling us
“Having backups run automatically is a far superior idea” is wrong. The problem with automated backups is that those disks MUST ALWAYS be turned on, with all the consequences such as premature wear-out, electrical issues, corruptions when systems crash etc.

This is what I consider safe and prudent: Have at least two or better three backup disks on which you make periodic (weekly) complete system backups. I use 2.5" disks that do not need their own power supply. Make the backup and then immediately remove the disk and store it in a safe place. Have the two disks (preferably from different disk manufacturers, so that you don’t run afoul of the “Death Star” effect) in different places (and I also use two different, but well-respected backup programs). Use additional, automated systems to cover the (relatively short) time spans between these manual backups.

A disk that’s always running is just subject to too many things that can go wrong: Several times I postponed manual backups when there was a thunderstorm in the area. Wear-out: None of my manual, stowed-away, backup disks has ever failed, but I cannot say the same of my time-machine disks. They’re always on, they always have to do work, and they wear out.

To me, manual backups are the long-term backbone, and automated systems can cover the rest.

A funny, but insightful look at backups (on the blog, you seem to have to press “previous” to get to the next principle):
http://www.connections.com/blog/21-it-backup/100-zen-and-the-art-of-it-backup-principle-1

1 Like
2

:mrgreen: :smiley:

3

The sticky was locked as primarily informational, although your comments are welcome. I’m planning to add another sticky about the various kinds of backup built into DEVONthink.

I agree with your points, except that most of them don’t apply to the way I work. :slight_smile:

That’s because I do my database work on a MacBook Pro Retina with 15" screen, 4-core i7 CPU, 16 GB RAM and 512 GB SSD… As it’s battery-powered (even when connected to a charger), it filters out power fluctuations and outages. My Time Machine backups are to rotated USB 3 portable drives that are powered by the MacBook Pro. Those drives do not run constantly when mounted. They sleep after a period and of course when the computer sleeps or is shut down. They do wake up when called by Time Machine. I’ve never had a portable drive failure, but I “retire” drives for backup use every couple of years, knowing that drives will eventually fail.

In 2005 I had some DEVONthink Pro projects that were beyond the capacity of my TiBook laptop, so had to work with desktop Macs. Ironically, one of the most demanding projects became an evaluation of the impacts of hurricanes Katrina and Rita on the health care infrastructure in Louisiana. I moved to a Mac Pro for added memory and storage capacity. The power grid stayed unstable for some time, so I had to buy an expensive uninterruptible power supply, which provided enough time to complete a process and shut down gracefully.

In 2007 I moved to a log cabin in Brown County, Indiana. It’s a great place to live, but when thousands of tourists come to see the scenery or there’s a storm, power outages are common. As Mac laptops could now handle most of my DEVONthink work, I moved to using one. I have a 27" iMac (also with i7 CPU) but rarely use it except for photo editing and display – for which it’s a great machine. My MacBook Pro handles power outages without complaint (I’ve got a backup generator), and when I go on travel I just pick it up, without complications such as synchronizing with another computer. (I also bring along a portable drive for Time Machine backups.)

“RAM is good; more RAM is better.” How true. I expect Apple will increase maximum RAM in laptops to 32 GB, down the timeline. I’ll be a customer. :slight_smile:

4

It is odd that your Time Machine disk is always spinning. I am sitting at a computer right now hooked up to a Time Machine disk and the disk is not spinning. If it is spinning all the time that might point to a problem somewhere.
I have no qualms with Time Machine. It is an important part of my backup strategy because it is fast, free, and convenient. But you are correct, any hard drive can fail (regardless of whether it is spinning continuously or not), and if your only backup is on a drive that fails, and your main system drive also happens to fail, you’re toast.

But manual backups do not necessarily prevent this problem except insofar as you might be extending your backup drive’s life to a small extent (insofar as we can predict with any accuracy the life span of a givin individual drive, which we cannot). It is still conceivable to have a failure of your manual backup drive and your main system drive.

I don’t necessarily feel that this is a debate about the suitability of manual or automatic backups, but instead is further justification for having multiple backups using multiple methods. I rarely do manual backups because I find them cumbersome. That does not mean I will, with certainty, fall victim to one of the issues GG378 points out. I have multiple backups, all of them automatic, with two on-site and one off-site. The off-site is hosted by CrashPlan so drive failures aren’t an issue. Yes, there are other issues with cloud backups, but that’s why I have local backups too.

That said, I most certainly do make manual backups at certain important moments such as if I am making some major change to my hardware or operating system, or if there is some other reason to do so. These are normally clones of my entire system to facilitate a swift recovery should something go wrong, rather than being some long-term backup solution.

The take-home point is, make sure that you employ multiple backup strategies that fit your workflow (I, for example, need relatively regular backups since I do a lot of writing. even a daily manual backup would not protect against massive data loss as I can do a lot of important writing in the course of a day). Make sure that these multiple strategies aren’t vulnerable to the same failures. Verify the integrity of each of those backups regularly.

In my system, a lot of very different issues would have to occur simultaneously to wipe out my main system and all of my backups. The chances of all those things occurring simultaneously are sufficiently low that I am not terribly concerned. Adding a regular manual backup to my current system would add a lot of inconvenience without adding much security.

So if your system includes a manual backup, that’s fantastic, but I do not agree that a manual backup is an absolute necessity to ensure the integrity of your data. I also disagree that a manual backup is inherently “bad” or “stupid” as suggested by the article. It is a perfectly fine compliment to a broader backup system.

5

Speaking of data loss: After typing in a pretty lengthy response, the internet connection went down, and took my post with it. Bummer!

That was sloppy language. For a typical TM setup, I mean “once an hour” (I changed the recurrence time to 4 hours on my TM). I might have a special situation. My wife’s MBP only writes typically a few MB per hourly backup, so the TM disk is not on for very long. I keep numerous things (e.g. my mail folder) in encrypted virtual disks (if someone steals my MBP, it will typically be in sleep mode; when they try to get at my disk by externally booting etc., those disk images will no longer be mounted; I prefer that to FileVault, but let’s not get into that). Since the sparsebundles are 8 MB stripes, a single 10 kB email added or moved to a different folder, then causes at least 8MB of TM writes. So my TM backups are relatively time intensive. On an hourly schedule, the disk would work 25% of the time at least. But let’s not worry about disk wear. I agree that that’s not the main concern these days.

Most importantly, an “automatic backup” disk is always on the bus and on the grid (and multiple TM disks that are rotated in an out are for the purpose of this discussion essentially “manual”, in the sense that lightning and filesystem corruptions do not take out these disks at once). We agree that we need MULTIPLE backup disks. But for manual disks, that means that they have no “common mode” failure point (other than creeping, unnoticed file corruption that propagates through your system, unless you regularly “retire” system backups and never touch them again). Multiple automated systems are still by definition always on the bus and grid, so common mode failures are a real danger (unless these systems are separated spatially and connected via a speedy network, but among individual users, that’s not the norm).

While I never had a problem, the forums are full of TM corruption horror stories. TM is extremely convenient, and that’s why I use it, but I would estimate (I pull this out of thin air here) that the reliability of a TM backup in the long run is an order of magnitude or more down compared to manual system backups with quality software. Never heard of corruptions via Synchronize Pro, Carbon Copy Cloner or SuperDuper (again, unscientific folklore on my part). And yet I do not trust a single one of them and use all of them on my different manual backup disks (all made by different hard disk manufacturers). You see, I’m truly paranoid!

Overall, the key is “multiple strategies at once”:

Automated: to catch everything regularly, fine-grained versioning of evolving files
Recurring manual (periodic overwrite): best chances of data survival
Retired manual or (confirmed uncorrupted) TM drives: avoid perpetuating unnoticed file corruptions
Online data storage: very orthogonal to your own backups

6

I don’t have enough knowledge about your setup, but basically you seem to have 2 on-site automated systems and one off-site (Crashplan).

If the automated on-site ones are as I define them (i.e. always on the bus/grid), and assuming a single building or even apartment as the “site” (we are presumably not discussing corporate solutions here), then:

-a single lightning strike
-an on-site fire
-a break-in (“always-on” disks cannot be hidden very well!)
-a computer corruption issue that messes up your main disk, and send bad data to both automated systems before you catch the problem,

will after a SINGLE incident reduce your options to a SINGLE REMAINING solution, Crashplan. You never want to be in a situation where only one backup is all you have left. I’m not sure on what kind of pipe you are sitting on, and how much data you have, but for me it would take a long, long time to get that data back via the net (or can you pay them to fill a disk with your data and express-ship it to you?). At that point I would be extremely nervous about the only remaining backup being completely out of my hands. So, no, I do not think that this scenario justifies considering a weekly manual backup (and transfer to a safe , separate location; and again “manual” here includes rotating two or more TM disks) as too cumbersome.

I would consider a minimum of 2, independent, net-based backup services as prudent.

Professional outfits still put things away on tape. They know why.

In my case, I only use a single laptop for all my work (other than big number crunching machines at work, but again, that’s not the scope here). Here is what I do:

  • The laptop goes wherever I go.

  • At home:

    • Automated TM backup
    • weekly manual backup on two different disks (Synchronize Pro)

  • At work (about 5 km away, remedies fire, break-in, most likely lightning strike, and flooding)

    • Automated TM backup
    • weekly manual backup on one disk (Carbon Copy Cloner)

  • Monthly research trip to a nuclear lab 2000 km away (protects against a thermonuclear weapon and against all but the largest meteorite impacts :slight_smile:

    • manual backup on one disk (SuperDuper)

All my data fits on a single 2 TB 2.5" disk (very easy to hide away). Starting this year, I will regularly buy new ones, and retire old ones (i.e. not update the backups) twice a year and store those (distributed evenly among the 3 locations; I have not addressed the issue of potentially “exercising” those old disks yet). That’s an annual investment of less than $300 for all my work and personal data. The “cumbersome” manual backups are second nature to me, like putting on a seatbelt. I don’t even think about it.

7

I am reviving this thread because I am only making backups using Time Machine and have recently began to think that I want to have a clone of my entire laptop on a cloud and several other methods.

The recommendations above are dated (CrashPlan, etc.), so I am hoping that someone can recommend current options that I can use.

I would rather not pay a subscriptions service for storing my data online, but the thought of losing it all to theft or fire or flooding makes me want to spend the money. All my backups right now are on TimeMachine on external hard drives. I have some files in Dropbox but if someone breaks into my place and takes my computer and external hard drives or a natural disaster occurs that damages my equipment, I am in huge trouble.

8

We often advocate Arq for online backups of important data, as they use a snapshot style backup that is safe for DEVONthink databases.

1 Like
9

I’ve been happy with Backblaze for over 10+ years, cheap and works well, they just added one month of free history so you could retrieve files that have been changed up to month of history.

1 Like