Incremental backups using rsync with bonus backup to amazon glacier storage

I’ve been using Crashplan for backups for 3 years and highly recommend their service. If you want set and forget backups of your important stuff, go for it. But for a few reasons I’ve switched to running my own incremental backup system, namely:

  • The cheap 3 year deal I signed up with ran out, so to continue I’d have to pay $9 a month.
  • Crashplan doesn’t work very well with encrypted home directories in linux (ecryptfs) being unmounted at boot time, and deletes your home directory from the backup.
  • To get around this you generally set Crashplan to backup /home/.ecryptfs instead of /home/user, but in the process you miss out on Crashplan’s ability to restore arbitrary files (you have to restore entire backup to pull out one file).
  • Crashplan’s attempts to compress and encrypt data that is already encrypted wastes cpu time and makes it take longer.
  • Storage on amazon glacier is cheap as chips – 1 cent per GB per month

For these reasons I set about finding a new backup system and stumbled on Rubel’s snapshot method. It works a treat but wasn’t easily configurable or installable, and doesn’t backup to amazon glacier, so I set about tinkering with it. I’ve released the end result of this tinkering as lincremental. Lincremental does incremental backups that you can access anytime, and if you are using ecryptfs like me, just do an ecryptfs-recover-private when you want to pull out a file from backup. As a bonus it also has a script to upload the latest daily backup to amazon glacier once every 28 days (configurable). I had planned on making incremental rather than wholesale backups to glacier but ran out of time to devote to the project.

It seems to be working fine and dandy for me, but it needs a lot more testing. Don’t blame me if you lose all your stuff. Use at your own risk and check the issues on github.

Metadata retention

Without the content of a message, you’re privacy can’t be breached, can it? What the hell is metadata anyway? Thankfully the Attorney General has posted a clarifying letter which makes it quite clear they will be pursuing the EU model (which has already been declared unconstitutional in Germany, Romania and the Czech Republic).

The metadata applies to:

landline, mobile telephony, internet access, internet email and internet telephony.

And the metadata contains:

  • the source of a communication
  • the destination of a communication
  • the date, time and duration of a communication
  • the type of a communication
  • the users’ communication equipment or what purports to be their equipment
  • the location of mobile communication equipment

Let’s break this down. Millions of Australians have smart phones which are constantly using the internet to check for new email, download updates, synchronize calenders etc. Every time this happens, their location (presumably which mobile phone towers they are connected to) will be monitored and stored.

If this model is followed, the minute to minute movements of millions of Australians for two years will be monitored and stored. If that isn’t a gross invasion of privacy, I don’t know what is.

Edit: Rodney posted the video below which tells the story better than I ever could.

Gone in 60 seconds – How to bypass data retention in under a minute

If the proposed expansion of national security powers and internet surveillance comes to pass, many people will want to bypass it to protect their right to privacy. Here’s one way to do it.

  1. Go to
  2. Pay $40 for a yearly account
  3. Download and install their software
  4. Log in to your account and connect

That it, you’re done in less than a minute. All your internet traffic will be encrypted and routed through their servers, bypassing any data retention by the Australian government.

Of course this raises the question, if it can be bypassed so easily, for $40 a year, how could the system possibly hope to catch determined criminals? I challenge the scheme’s supporters to provide an answer.

Send Nicola Roxon a copy of 1984

Given the recent push for increased surveillance, ISP data retention, and the erosion of any “right to remain silent”, I wonder if Nicola Roxon has read 1984.

First edition cover of George Orwell's 1984

Let’s make sure she has, or at least that she re-reads it, given the circumstances. I’m going to buy her a copy and send it to her address at parliament house:

Attorney-General Nicola Roxon
Parliament House, Canberra 2600

You can do the same. Find a copy in your local book store, or order online at fishpond ($9.51 including shipping), book depository ($9.43 including shipping), or amazon. Be sure to mark it as a gift and include a note, but be civil. We are trying to convince her that our privacy and freedoms are important, and that new powers will violate them in an attempt to catch criminals that has dubious efficacy at best. Twenty copies of 1984 landing on her desk should get our message across.

If you send one, let me know in the comments so I can tally it up. If you’d like to remain somewhat anonymous, I could send one on your behalf, just contact me on email at, using my public key for encryption of course. I promise not to retain your data.

Turgid Waters

Here’s a gem of a poem I found going through some old backups:

The water swirls, the paper furls.
Faeces drown in turgid waters.
And yet, their essence lingers.
Dragged kicking and screaming
To the locker of Jones,
the stool exacts its revenge-
Skidmarks mar the bowl,
Sights of poo’s last plight
To remain above, in the world of light.

I have the power to erase it.

Taking in hand the cleansing rod,
I scour the bowl and make it clean.
Bacteria’s last bastion burned,
Footholds falter, welcome spurned.
The pure porcelain smiles,
Like the Cheshire Cat,
Laughing at waste gone down the rabbit hole.

I wrote that in response to the toilets at my old work always being disgustingly shit stained, and hung it on the back of the stall doors so that people would read it mid act. I’m not sure if it led to any cleaner bowls but I’m sure it got a few smiles.

Super thanks to the folks at Crashplan without which this missive would have been lost for all time (along with that old gnupg private key I just revoked).