Backing up with Duplicity, Effortlessly

Duplicity has a wonderful design feature: it’s really really simple.

Duplicity has an annoying design flaw: it’s really really simple.

In case you didn’t notice, Duplicity‘s simplicity is both helpful and inefficient. It’s extremely easy to start using Duplicity because its usage is so very simple. Want to back something up? Just write “duplicity /some/directory ftp://user@host.com/some/other/directory” for FTP backup, or “duplicity /some/directory file://some/other/directory” for local backup, or any other protocol out of the 11+ protocols it supports. All it does is write files to the location of your choosing, and then it can recover and list files (either latest or at a date of your choosing) from the backup directory you give it.

This simplicity is really great for the simple use-cases, like backing up a home directory. It’s when you get into databases, exclusion and inclusion rules, and other such fine print that you have to plan a little.

I’ve got three different “things” I have to back up: my web development stage, my remote shell (mainly for irssi), and my home directory. Each one presents its own challenges, which I break down below.

The Secret to Backing up Home Folders

Here are the things I do to keep my home folder backups to 2 GB or less:

• Clear Firefox’s cache. This is a big space hog, and cache is worthless.
• I filter out “.azureus/torrents/”, a folder of mine, so that any BitTorrent downloads aren’t included in my backups. They usually are demos of games I want to try before I buy, so I never keep them anyway.
• Any software or data I could care less about or can easily reconfigure I usually filter out as well. This includes:
  • ePSXe: barely used, never keep the games or save data
  • Vendetta: demo, why keep it?
  • Wine: because everything I do with Wine is easily deletable or easily redone
  • Videos I re-encoded for my Palm: would rather re-encode again than back up 1GB+
  • All ISO’s: can redownload
  • VirtualBox: can reinstall the virtual images much easier than backing up and restoring 8+ GB.

Duplicity’s command line for the operation, in all it’s glory, is this:

duplicity $1 -v9
–exclude “/home/jacob/.azureus/torrents/**”
–exclude “/home/jacob/.epsxe”
–exclude “/home/jacob/.vendetta”
–exclude “/home/jacob/.wine”
–exclude “/home/jacob/.feeded/vids”
–exclude “/home/jacob/Documents/Personal/Music”
–exclude “**.iso”
–exclude “/home/jacob/.VirtualBox”
/home/jacob ftp://USER@HOST.EXT/folder/folder

With these settings, my backups take less than 15 minutes a day. If this hasn’t convinced you to start a regular backup, I don’t know what will. (And if you’ve already tried, but find it hard to get into a habit, and don’t like cron because you don’t know of a good time when your computer is on and you’re not using it, try doing your backups before you log in in the morning.)

Duplicity Backs Up Beautifully (And Anywhere)

After struggling with what good backup system would work for me, I’ve finally settled on duplicity. It doesn’t have a user interface, and it doesn’t even use cron (though you can manually punch it in). It does, however, do exactly what I want: It can tell me what my system looked like at a particular date, because it tracks deleted files as well. Not only that, but it compresses and encrypts everything into what appears to be a custom tar format.

And, unlike most other backup systems, it can back up onto FTP, or just about any other storage medium for that matter, because it doesn’t use fancy links or any other junk. It uses plain ‘n simple manifest files.

Duplicity is something you can have working within minutes if you can use a terminal. Granted, you have to manually back everything up, but that’s good for me, since I don’t really have a standard schedule of when I’m on and off the computer.

Give it a try some time.

Powered by ScribeFire.