Mike Rubel

Still one of my all time favorite approaches is the DIY-style snapshot-based system with rsync and hard links, which keeps snapshot sizes small: This example keeps the days worth of filesystem backups: rm -rf backup.3 mv backup.2 backup.3 mv backup.1 backup.2 cp -al backup.0 backup.1 rsync -a --delete source_directory/ backup.0/ More details: http://www.mikerubel.org/computers/rsync_snapshots/ Trinkup is a script that automates this approach, somewhat like rsnapshot: https://gist.github.com/ei-grad/7610406

There is a decade old guide to using rsync for backups and snapshots at http://www.mikerubel.org/computers/rsync_snapshots/ which I originally based my hand-rolled arrangements on. It hasn't been updated since 2004 but is still relevant. There are tools that make this more hand-holdy if you prefer to do less work/thinking yourself, like rsnapshot. For extra safety against hack+delete+ransom attacks, I make sure my backup servers and main kit have different credentials can't talk directly to each other at all - this way if someone hacks into my mains they can't automatically get at my backups and vice versa. I have an intermediate backup location: the active machines push data to that and the backup services pull from there, it can't login to either of the other sets. For automated backup testing, which I recommend you find time to setup, some data goes the other way (backups push to intermediate, other sites pull from there).

What Lutus talks about exists even when it's not for anyone else. I have never been paid to write code, and my formal CS education is limited to AP Computer Science, and a one-credit Java class in college. I wrote 20 years ago a backup script implementing Mike Rubel's insight <http://www.mikerubel.org/computers/rsync_snapshots/> about using `rsync` and hard links to create snapshots backups. It's basically my own version of `rsnapshot`. I have deployed it across several of my machines. Every so often I fix a bug or add a feature. Do I need to do it given `rsnapshot`'s existence? No. Is it fun to work on it? Yes. (I've over the years restored individual files/directories often enough from the resulting backups to have reasonable confidence in the script's effectiveness, but of course one never knows for certain until the day everything gets zapped.)

The original post that introduced this idea into general public: http://www.mikerubel.org/computers/rsync_snapshots/ I’m sure others will chime in that they used hard links like this before then, however as noted in that page, it’s the one that made it popular enough that rsync was updated to support the idea natively.

I use a rsync script that does incrementals but keeps full copies of each backup. Although mine is significantly different, it is based on http://www.mikerubel.org/computers/rsync_snapshots/ by Mike Rubel.

I've been using this for the past 20 years and it's saved my bacon multiple times: http://www.mikerubel.org/computers/rsync_snapshots/

Sorry - that is in purpose ... The definitive page for rsync snapshots has been, and always will be, here: http://www.mikerubel.org/computers/rsync_snapshots/ ... we actually don't want people to do rsync snapshots anymore because ZFS snapshots are much more efficient and use up less of their rsync.net account. If you change one bit of a file, the rsync snapshot method will cost the entire size of that file, since it's a hard link and is either broken or not broken. But if you just do a "dumb" sync to us and change one bit in the file, your ZFS snapshot will take up just one more bit. So not only do you get a much simpler backup script - you can really just do a dumb sync to us and let us rotate your snapshots - but you get more efficient space usage for changed files.

Actually, I was thinking of rolling hourly/daily/weekly backups and I didn't understand how I would set that up with your program. Would I have to sync the same source to multiple destinations on different schedules? That seems inefficient compared to e.g. http://www.mikerubel.org/computers/rsync_snapshots/ Please don't get me wrong, I'm not trying to denigrate the software you've written! I was just surprised to see it marketed as a backup tool instead of a file synchronization tool. The git packfile format is neat; I'm sure you can get some inspiration from https://github.com/bup/bup (which uses it) and http://obnam.org (which has it's own format).

For Linux backups, you might be interested in this article, if you haven't already read it: http://www.mikerubel.org/computers/rsync_snapshots/ The combination of cron, rsync and hardlinks, makes for incremental backups with versioning and minimal storage space, all over ssh.

rsync isn't bidirectional (see unison: http://www.cis.upenn.edu/~bcpierce/unison/), but it can be hacked on to make it store snapshots (see http://www.mikerubel.org/computers/rsync_snapshots/, but it's a full volume snapshot, not a per-file versioned history).

Zimbra's open source edition's WebDAV sharing works well once you get the URLs figured out. You get a lot more fine grained permissions. It's cross platform, there are shared folders, etc.. if that's what you're looking for. You can make folders public to share with others, and so on. With bandwidth as fat as it is these days DAV's heavyness doesn't feel so heavy anymore. Am I the only one that shares these sentiments? Did I mention it works on MacOS 10.6, Linux, and Windows 7? If you're using Dropbox to back up some of these other solutions might make more sense. Crafting up a script using hard links and rsync can give you daily snapshots of any filesystem at only the storage cost of the delta (which can be easily turned into a cron job) .. here's a decent resource for that: http://www.mikerubel.org/computers/rsync_snapshots/

This isn't exactly true. A vigilant user will run normal backups, and those backups will not be in a location that is writeable by the unprivileged user. In this way, it will not be possible for you (or an attacker) to completely wipe out your important files, only those changes from the last hour or so. Based on an rsync-backup article that I bookmarked a long time ago (http://www.mikerubel.org/computers/rsync_snapshots/), I run backups of my important files (/home/, /etc/, /usr/local/, some directories in /var/, and so on) every hour to a /backups/ directory that is only writeable by root. Every day, I copy a backup over to another machine using the same rsync process.

This article is about backups for servers. I think some people are missing that part. For my personal home machines, I was recently researching backup systems. I decided against cloud backups and opted for the simple solution of a USB 3.0 drive. It's cheap, and if I ever need to get my backup quickly, I just pick it up and take it. For critical documents, I can't imagine most people have more than a few megabytes anyways. That stuff can be archived/encrypted periodically and sent to cloud email (gmail, yahoo) much cheaper and easier than worrying with S3. For the actual backups, I'm using Linux so I went with rsync. See here for roughly how it's done: http://www.mikerubel.org/computers/rsync_snapshots/ The USB drive is encrypted, I then tell rsync to use hard links and it does the incremental backup. The nice thing about hard links is you can get at any file you want to, in any backup you want to while preserving space. Files in new backups that exist in prior backups are simply hard links. Then you just "rm" older backups when no longer needed. It's an incredibly elegant solution on Linux, especially if you combine it with LVM snapshots.