disengage.ca a quest for the technomadic lifestyle

CloudCamp Aftermath

Wow, that was great! It's so nice to finally be interested in a technology again - I was honestly starting to think that I'd never enjoy another conference. Half of the fun of a conference is discussing new concepts and ideas in technology, but the other half is meeting up with folks who have similar ideas and interests, both on a professional level but also in a social (ie: beer) environment.

It was also my first "un-conference". The idea behind this is that there's a schedule, but none of the talks are booked in advance, and people come with a talk or presentation that they'd like to give and sign up to do so on the spot. In the end it's more about discussions than presentations, and in fact I was drafted (or more accurately, "tricked") into hosting a discussion group on scaling Drupal in the cloud.

Now *that* was an eye-opener - I had naïvely assumed that most people working with cloud computing were working with web applications, as that's what most of the documentation out there seems to be. Or maybe I just pitched my session wrong, and should have stressed all web applications and not just Drupal. Or perhaps I shouldn't have scheduled the discussion at the same time as Dan Kaminski's discussion on cloud security. Regardless, the room began with approximately eight people, and as it became apparent that we were really looking for a technical discussion but most of the people were in marketing and management, the room shrank to four people.

What was awesome was that the four people were:

Wild! I got more out of that half-hour session than I have reading documentation for the past month and a half. As the session started, we basically went around the room discussing backgrounds and technical histories, and it became apparent that I was really the only person in the room with any technical experience at all working with the EC2 cloud, I began to present my experiences to date, focusing on the three big problems that I had butted my head against - how to "auto-scale" web front ends using load balancers, how to scale MySQL databases elastically, and how to share storage between EC2 instances. As I discussed these, Dean Dierickx from RightScale showed up, and had some interesting notes to add from a high-level design perspective, which we talked about at length. We then started discussing database scaling, and it became apparent that nobody in the room had any practical experience with that at all.

Right about then, Jonathan Lambert from Work Habit let himself in, and after listening to us fumble about for a couple of minutes, grabbed a whiteboard marker and launched into a fifteen minute in-depth technical discussion on different methods of scaling MySQL. This was *great* stuff, though it pretty much cleared the room of the marketing and manager types. As he finished, he validated my whole trip with one statement - he said something like:

"Now, there are three major problems that everyone attempting to scale any web application in the cloud butts their heads against, and there's no simple answer to any of these: how to launch instances automatically from a load balancer, how to scale MySQL, and how to share storage between instances. We've fought these problems hard for two years now, and we still don't have a good answer, but here's how we've managed to get everything working so far..."

...and proceeded to lay out a basic, scalable platform for Drupal on the whiteboard. Most of his layout matched my work exactly!

To top that off, just last week I downloaded a script from the RightScale website that purports to set up a basic EC2 instance to be a "RightScale" machine, and poked through the script to see exactly what they're doing to prep a machine. I was shocked to see that every step of the way, their work matched what I was already doing to setup my own EC2 instances! That may not sound like much, but frankly after fifteen years of Linux administration, I have some particular ideas about how a machine should be setup for optimal networked administration; which additional packages should be installed, which default services should be turned on or off, what changes should be made to the default shell environment, etc. Seeing two large, successful operations doing exactly the same things as I do in my personal environments, and facing exactly the same challenges, was vindication to say the least!

Anyhow - even though it was a "free" conference (with the quotes of course implying bus fares and hotel fees), I think I got well more than my money's worth.