Head in the Clouds
Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.
Portability: No snowflakes for you!
An author, whose opinion I respect, wrote recently about how cloud portability 'is a bit of science fiction' at this point. While he is technically right - the idea of moving running machines from a CloudStack cloud to a vCD cloud, CloudStack, or AWS is still at best 'a bit of black magic', and in other cases simply isn't feasible at all. And doing it at scale??? that's just crazy talk. That said, I'd argue (yes, this is largely just my opinion) that in most cases if you are trying for portability at the Infrastructure-as-a-Service layer, you are doing it wrong.
You see, this 'problem' of portability isn't a new one, and really isn't even cloud related, I think it's far more fundamental than that. I remember, in my days as a Pimply-Faced-Youth, of keeping exact copies of machines as cold spares - the thought being that we might need to migrate the services living on one piece of hardware, and particularly for a certain non-Unix operating system, restoring a copy of a disk, or even the disks themselves to different hardware often meant things just wouldn't work. (sound familiar?) Sometimes you got lucky and it would, and sometimes you could coerce things into a working state by installing new drivers, or updating initrd, or even adding kernel modules. So when buying hardware, we'd buy two of everything (or at least N+1) for those services that were truly business critical. Yes it was incredibly expensive, and wasteful, but it mitigated risk.
A bit later in my ops lifetime, virtualization became mainstream. Finally - we had this psuedo-hardware that would look the same regardless of the underlying physical hardware - hypervisors like VMware and Xen did wonderful things for us. Admitedly we couldn't easily migrate between hypervisor types, but life was still much better off. Not only were we able to stop buying so many 'spares', but we were a bit less concerned about machine lifecycle and had much better utilization to boot.
Folks came a bit further along with tools to convert these virt machines from VMware to KVM or Xen. As a matter of fact, on a previous blog I used to maintain, one of my most-viewed articles was one that detailed the use of qemu-img and other tools to convert VMDK files to raw disk images. I did more of this than I care to admit - and honestly when I did it, I was striving for portability, even if in a kludgy, barely automted way. Portability was the wrong thing to seek after - and I should have already known it at that point in my career.
Portability is the equivalent of trying to preserve a snowflake. It's difficult, usually messy, and often ineffective. What I didn't realize is that I really shouldn't have any snowflakes - that what I was really after was a way to consistently reproduce, not a system to save the only copy the world would ever see. Of course configuration management already existed, why I didn't understand the problem and the solution that was already out there I don't know. Perhaps I was a latent server-hugger, or feared obsolence by a few lines of ruby - but configuration management (coupled with automated provisioning) meant I didn't need portability - didn't even want portability.
You see we really are at a point where a virtual machine is a commodity. Yes hypervisors and cloud platforms aren't quite at commodity level, many of them do things that are still quite novel and interesting, but at the lowest level, the VMs they offer up certainly are commodity. And as a commodity (a largely renewable commodity at that) it's easiest just to harvest them and redeploy. I'd even go so far as to say it's preferable to redeploy. If my 'known good' state is in configuration management, destroying the machine and redeploying it is trivial, I can confirm that the machine is in the correct state, that no changes outside of my configuration management scope are the cause of the problems. Because I can use the same definition of 'Just enough operating system' (JEOS) in all of my deployments (regardless of whether they are bare metal, VMware, AWS, or any other of a number of platforms, I know I can start with the same base, and then use configuration management to render my mass-produced server instance perfectly - regardless of environment.
There are other problems with portability (bandwidth constraints among them) but I am convinced that as sysadmins, we no longer want or need portability. We want replicability, the server instantation factory. If we revert back to collecting snowflakes, we'll spend all of our time trying to preserve them, and inevitably end up failing along the way.
 Black magic for most - I know that Euca says they can move AMIs between AWS and Euca and that it works for most of the images out there.