Mac GeekeryGet your geek on. |
|
blog advertising is good for you
recent popular content
User login
|
As others have noted, there’s nothing all that new about Time Machine other than its UI. UNIX admins have been doing similar incremental backups for ages immemorial. Well, OS X is a UNIX, isn’t it? So it should follow that we can get the same bang-for-buck in Tiger (and possibly prior versions, too). For this, rsync and cpio are our friends today. Adam Knight did most of the footwork here; I just hacked it up into OS X-compatibility. The OriginalThe original script1 comes to us from Mike Rubel, as linked in the footnotes. The script he provides is pretty specific to Linux, though, and assumes the GNU cp (which we do not have) and a few Linuxy paths. I’ve cleaned up the script to point to the correct paths in OS X. BSD cp doesn’t do some of the same magic as GNU cp, so it also takes a bit of working around to get the same effect. Specifically, the GNU version allows you to copy as links instead of creating a new whole copy of the file, whereas the BSD version does not. Never fear, after a fashion, cpio can. OursWhereas GNU cp allows us to do this: $ cp -al foo1 foo2 In a BSD environment, we get to do this: $ cd foo1 && /usr/bin/find . -print | cpio -dplm ../foo2 The means are not as elegant, I know, but it accomplishes the same ends. I also removed the root user check, as well as some volume mounting and remounting work that don’t apply to OS X (or at least, not for our purposes). In the end, we have: #!/bin/bash Change the paths for $SNAPSHOT_RW and others as per your needs. As is, this will create four revolving snapshots (one “current” and three historical). Lather, rinse, and repeat the sections with mv commands to create a longer chain, if you wish. This is automated easily by simple inclusion in your crontab. Rubel provides for hourly, daily, and weekly rotations with a few other scripts, which you can see at his site. Similar changes could be made to those scripts to effect the same results. I leave that as an excercise for the reader. 1 This tip is derived from Easy Automated Snapshot-style Backups with Rsync and Incremental Backups Using Rsync.
About JC
Author Biography JC is a former Mac Genius and Mac-centric IT worker with a background in print advertising. He earned a reputation as a miracle worker when he saved the day at a new business pitch with the arcane knowledge that Apple’s ADB cables were nothing more than poorly shielded S-Video cables. JC runs the Heroic Efforts Data Recovery Service and writes Ungenius, a tawdry tale of the life and times of a former Mac Genius. You can contact JC via IM or via the contact form. |
Isn’t the whole point of Time Machine that it leverages zfs, the filesystem from Solaris, that has built-in revision control?
While it’s useful to have these tools, they solve a fundamentally different problem.
Time Machine has nothing to do with ZFS; Time Machine is a fancy UI on top of snapshots of some sort. This is exactly that…. without the fancy UI.
If you want to use GNU cp for this, you can grab the source or a binary of it and keep it in another place on your drive for this purpose (/usr/local/bin or /opt/bin or something). I recall Fink having a package for it, but I can’t find it at the moment.
I don’t quite understand the part of this article about hard linking files. I want to throw this in the hopper, however, in case it’s useful.
I’ve been using a script similar to this for a few years to back up some data. The primary difference in the script I use is the –link-dest option in rsync. I create a folder for each day’s backup, like the script in the article, but the –link-dest option instructs rsync to hard link existing files in the current backup to the old, and only write new data.
In other words, if a file exists at the backup destination, rsync creates another entry to that existing file in the file table that points to the same data, except in today’s backup folder. I know that’s a monster sentence, but I’ve re-read it a few times and it does make sense.
In any case, you can man rsync and read about –link-dest if I’m not clear.
Here’s my code:
That’s basically what my script is doing, using a slightly different method, and with a ton of safety/sanity checks in place.
Hi, interesting solution. I didn’t try it out yet but just reading the script led me to think: isn’t the ‘mv’ statements wrong? I mean, you’d like to shift the backups ahead, 5 to 6, then 4 to 5, then 3 to 4 and so on, but in the above script you’re shifting from 5 to 6, then 5 to 5, 5 to 4.
Am I missing something? If not you probably meant:
Is that right?
You’re correct. That script is incorrect because I had some problems using the code tag (line breaks didn’t show) so I tried to cut and paste. I’ll edit it.
Why make it so rigid?
I get the following error on the initial run:
Seems to me that you need to create directory “snap.0” before doing the first backup. Might want to add that to the script by altering step 3:
works without the mkdir just fine for me…. what did (exactly. to the character. it’s important) did you set SNAPSHOT_RW to be?
If you use rsync you might want to use —extended-atgtributes (or -E). Then rsync will also copy Mac-specific metadata.
I use rsync for all sort of synchronisation (iTunes/iPhoto data to multiple machines and such). Very powerful.
Why not just use software that follows this strategy all ready, with many more features and maturity? Like rsnapshot.. http://www.rsnapshot.org/ Though one drawback with these hard link approaches, is that the snapshot will contain complete copies of each object in the snapshot (just like traditional backup systems), so there could be a lot of bits that are being duplicated between snapshots.
For most backup strategies, I prefer the approach employed by rdiff-backup ( http://rdiff-backup.nongnu.org/ ). With this system, each snapshot is expressed as a collection of reverse diffs from the last snapshot. So, it can be be much more space efficient than traditional methods. It goes a step further by gzipping each diff. The tradeoff is increased time it takes to restore a snapshot, due to having to apply each snapshot diff, till it reaches the target date. But then, I think that’s a generally acceptable tradeoff, when it comes to backup strategies.
Just a note. ZFS uses snapshots to version stuff. Also Time Machine does this on the fly with files it seems, you do not kick it off. Also retrieval seems to be per file and fast.
Retrieval here is “per file” and “fast” too
find it, copy it back over. it just doesn’t have a slick UI.
and, just a note, ZFS is just a rumor. It stems from its inclusion in an intermediate developer build, but despite Sun’s CEO’s comments last week, it seems that ZFS may not be there at all anymore. At this point, there’s no telling whether it will ever ship from Apple.
Too bad there isn’t some way to get at the data that Migration Assistant moves. MA seems to do a pretty good job of getting all the data that is not present on a standard OS install and ignoring the rest.
Normally, MA imports data from another drive into the current system. Unfortunately, I don’t know of any way to get it to work backwards. In other words taking the kind of data that MA would normally import, but instead making a copy of it from the current system, to an external drive. (or other location)
Of course if that could be combined with some of the features of rSync, such as the use of links to save space, and the ability to have an rSync server, it would seem to be a good starting point for what one wants for backup.
Hello,
as i got no programming skills, can anyone please tell me what i have to change in the original script (steps 1 to 5 in the script itself) to only backup 2 versions (snap.0 and snap.1) of my data and not 4?
Thanx.
Time Machine can restore metadata, ACLs, permissions, extended attribs, meta, forks and inodes – rsync cannot.
While this concerns only a small percentage of people, it IS a concern and a big one for the people it affects. It would seem there are virtually no backup softwares capable of versioning for Mac OS X that copy ACLs, Unix perms, extended attributes, metadata/forks/inodes. And if one utility does copy.. say.. ACLs, it doesn’t copy forks (for whatever reason).
Super Duper is actually the ONLY backup software to copy absolutely everything, including all the extended file system stuff. Unfortunately, it is only really useful for image dumping… not for individual file versioning.
Time Machine might be the first program to actually allow versioned backups AND copy all the extras for Mac OS X. I think this is worth pointing out to anyone who plans on using rsync. It does not, for example, copy ACLs…
Not true. Rsync, by default, copies permissions and some metadata (dates). With the -E option, it copies forks, ACLs, and xattrs. Neither solution restores files to the exact inode they were on (this is impossibly difficult to do reliably).
Super Duper is not the only software to copy this information.
dittodoes it, which means Carbon Copy Cloner does it. As doesasr.Hey fellow geeksters! Wouldn’t it be much more savvy if we just wrote out a program for 10.4 that would execute all this for us automatically?
It’s probably as simple as an Automator app.