This is great but one thing bugs me. When you add a new user, you have to restart SSH? Wouldn't that prevent people from connecting every time a new user is added?
The video in your blog post makes this murky to understand. First, the screen says "Restart the server to apply any changes". Second, you enable Authy for root, and then restart the server and say "Remember to restart ssh so the new configuration takes effect". Perhaps I'm not linux-oriented enough, but for me "configuration" doesn't just mean system config - adding a user could be part of that definition.
Otherwise though, I do think that this is an awesome idea. It only needs apps for smarphones instead of sending an SMS and you're on part with Google and Battle.net :)
