Full disk encryption chaining with dm-crypt, cryptsetup and luks

In a fit of either curiosity or tinfoil induced paranoia, you decide to set up full disk encryption on your machine. But it’s really annoying because you have multiple physical disks, and you can’t be arsed entering passwords for each one separately at boot up. So what do you do? You stick a keyfile on the first encrypted disk, and decrypt the others with that instead of a password. That way they are “chained” together – the password decrypts the first disk, which unlocks the file to decrypt the secondary disks.

Here’s how you do it (works on debian wheezy):

  1. Encrypt all disks normally using luks/cryptsetup/disk utility
  2. Set them all up to be mounted at boot by fiddling with crypttab and fstab (arch wiki should have you covered)
  3. reboot and go through the tedium of entering multiple passwords
  4. generate yourself a new keyfile for the secondary drives: # dd if=/dev/urandom of=mykeyfile bs=512 count=8
  5. stick that keyfile somewhere safe on the primary encrypted disk (with admin read only access)
  6. add the keyfile to a luks keyslot on the secondary drives: # cryptsetup luksAddKey /dev/[volume] /path/to/mykeyfile
  7. fiddle with crypttab to make it use the keyfile on boot: [volume]_crypt UUID=deadbeef-dead-beef-dead-beefdeafbeef /path/to/mykeyfile luks
  8. reboot and test it

Be sure not to store the keyfile somewhere stupid where it will be unencrypted, like in /boot for instance. Bonus points for being patient and using a better source of randomness than /dev/urandom. The usual disclaimers apply, I don’t really know what I’m talking about, so don’t use this method to secure your nuclear launch codes, blame me if someone steals your data, or blame me if you can’t decrypt the drive and lose all your data.

Newspaper ownership in Australia

Today I spent some time hacking together an ownership graph of Australian newspapers. Spoiler: it’s all owned by News Limited and Fairfax, but click on the pretty picture below and you can see exactly how stuffed it is in an ugly interactive physics based format where you can click and drag stuff around.

Preview of newspaper ownership in australia

Arrows indicate ownership of course, with X–>Y meaning that X owns Y. You can see the code if you want. I might make it less ugly in the future but for now, it is what it is.

Incremental backups using rsync with bonus backup to amazon glacier storage

I’ve been using Crashplan for backups for 3 years and highly recommend their service. If you want set and forget backups of your important stuff, go for it. But for a few reasons I’ve switched to running my own incremental backup system, namely:

  • The cheap 3 year deal I signed up with ran out, so to continue I’d have to pay $9 a month.
  • Crashplan doesn’t work very well with encrypted home directories in linux (ecryptfs) being unmounted at boot time, and deletes your home directory from the backup.
  • To get around this you generally set Crashplan to backup /home/.ecryptfs instead of /home/user, but in the process you miss out on Crashplan’s ability to restore arbitrary files (you have to restore entire backup to pull out one file).
  • Crashplan’s attempts to compress and encrypt data that is already encrypted wastes cpu time and makes it take longer.
  • Storage on amazon glacier is cheap as chips – 1 cent per GB per month

For these reasons I set about finding a new backup system and stumbled on Rubel’s snapshot method. It works a treat but wasn’t easily configurable or installable, and doesn’t backup to amazon glacier, so I set about tinkering with it. I’ve released the end result of this tinkering as lincremental. Lincremental does incremental backups that you can access anytime, and if you are using ecryptfs like me, just do an ecryptfs-recover-private when you want to pull out a file from backup. As a bonus it also has a script to upload the latest daily backup to amazon glacier once every 28 days (configurable). I had planned on making incremental rather than wholesale backups to glacier but ran out of time to devote to the project.

It seems to be working fine and dandy for me, but it needs a lot more testing. Don’t blame me if you lose all your stuff. Use at your own risk and check the issues on github.

What next?

Having finished Decisive Robot, Ghost Safety Squad and releasing their source code, what do I work on next?

Decisive Robot hasn’t really taken off so I’ll probably leave it to it’s own devices. People seemed to like Ghost Safety Squad though, so I’m considering revamping it into something bigger and better. Make it more fun and less punishing, polish it up and chuck it on phones/tablets while I am at it. Or there is “mystery game” which I was working on before either of those two showed up.

I like the idea of Ghost Safety Squad but it’s the sort of casual game that appeals to a wide audience. This means it has to compete for attention in a market full of other awesome games, and it’s success will rely a lot on it’s graphical appeal and publicity – not my strong points. “Mystery game” is very niche and could play to my strengths as a software developer. It will probably take longer to develop though.

Everything I’ve been reading about entrepreneurship suggests that you’ve got to fail many times before you’ll make something people will really like. With that in mind it’s probably best to focus on small projects I can finish in a month or two, rather than devoting a year to something that will likely fail.

Hmmm. Oh crap, why am I debating myself in this ridiculous soliloquy when I created a robot to make these kind of decisions for me?

Decisive Robot chooses "Mystery Game"

Well that settles that.

FEELING INDECISIVE, HUMAN?

Life is full of decisions. Should I cook soup, burgers or pasta for dinner? Should I watch the avengers or spiderman tonight? In short, life is full of first world problems. I have created a first world answer:

DecisiveRobot.com

Let Decisive Robot take the burden of indecision away from you. Ask it your questions and get an immediate response. Here is an example:

Decisive Robot isn’t very smart though, if you try asking it “What should I do with my life?” or “What is the flight speed of an unladen African swallow?” it won’t be able to make sense of it. Stick to either-or options such as “Should I go for a walk, eat a peach or just sleep in?

The bot is also on twitter, where you can tweet your questions to it and it will answer. To make sure you see the responses either follow @DecisiveRobot or look at your connect tab on twitter. The bot should answer within 30 seconds.

To university or not to university, that is the question

If I was to start again, as a fresh faced high school graduate, I would not go to university.

Well okay, maybe that’s not true. I might go to uni, but it wouldn’t be for the first rate education, it would be to bum around for 3-4 years, meet people and have fun. I work as a programmer, and there are many problems with tech courses at university:

  • Tech moves fast, universities don’t. The syllabus is always a decade behind.
  • Assignments are extremely short term projects. You don’t have to live with your mistakes or work effectively in a group.
  • At the end of your course, you’ll have a shiny piece of paper, a large debt, and nothing show people or employers and say “I made this!”

In contrast with university, online resources for learning have gotten dramatically better:

  • Think Python is a great book on learning the fundamentals of computer science.
  • Khan Academy’s computer science section is great.
  • Stack Overflow is a great resource for asking questions (and answering them, to build a reputation).
  • Github is the one stop shop for hosting your projects, showing off your work, and collaborating on volunteer open source projects to learn and prove your skills.

The main downside to not having a university degree is that some employers might not consider you if you don’t. I can’t speak for others, but when I look at people’s resumes (which doesn’t happen very often admittedly), I don’t even consider it. And this is coming from someone with a first class honours degree.

But it’s not all bad, I can see some advantages to university. If I’d tried the self education approach when I was younger, I might have slacked off a lot. University gives you some focus and direction (aka deadlines). There’s also an emphasis on written communication at university which always comes in handy. Not to mention actually getting feedback (aka grades) on your work and progress. And the whole physically-present-with-other-like-minded-individuals thing, as opposed to being a lone coder in your mother’s basement.

One thing is for sure though, you need more than just a degree. Work on a project outside of your studies, post it online for all to see. Contribute to an open source project of some kind, or start your own. Make something. That’s where you’ll learn the most.