satis egitimisatis egitimitengda.pro

Open@Blog

Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Login
Posted by on in Cloud Strategy
  • Font size: Larger Smaller
  • Hits: 59883
  • Print
  • Report this post

Portability: No snowflakes for you!

An author, whose opinion I respect, wrote recently about how cloud portability 'is a bit of science fiction' at this point. While he is technically right - the idea of moving running machines from a CloudStack cloud to a vCD cloud, CloudStack, or AWS is still at best 'a bit of black magic'[1], and in other cases simply isn't feasible at all. And doing it at scale??? that's just crazy talk. That said, I'd argue (yes, this is largely just my opinion) that in most cases if you are trying for portability at the Infrastructure-as-a-Service layer, you are doing it wrong.

You see, this 'problem' of portability isn't a new one, and really isn't even cloud related, I think it's far more fundamental than that. I remember, in my days as a Pimply-Faced-Youth, of keeping exact copies of machines as cold spares - the thought being that we might need to migrate the services living on one piece of hardware, and particularly for a certain non-Unix operating system, restoring a copy of a disk, or even the disks themselves to different hardware often meant things just wouldn't work. (sound familiar?) Sometimes you got lucky and it would, and sometimes you could coerce things into a working state by installing new drivers, or updating initrd, or even adding kernel modules. So when buying hardware, we'd buy two of everything (or at least N+1) for those services that were truly business critical. Yes it was incredibly expensive, and wasteful, but it mitigated risk.

A bit later in my ops lifetime, virtualization became mainstream. Finally - we had this psuedo-hardware that would look the same regardless of the underlying physical hardware - hypervisors like VMware and Xen did wonderful things for us. Admitedly we couldn't easily migrate between hypervisor types, but life was still much better off. Not only were we able to stop buying so many 'spares', but we were a bit less concerned about machine lifecycle and had much better utilization to boot.

Folks came a bit further along with tools to convert these virt machines from VMware to KVM or Xen. As a matter of fact, on a previous blog I used to maintain, one of my most-viewed articles was one that detailed the use of qemu-img and other tools to convert VMDK files to raw disk images. I did more of this than I care to admit - and honestly when I did it, I was striving for portability, even if in a kludgy, barely automted way. Portability was the wrong thing to seek after - and I should have already known it at that point in my career.

Portability is the equivalent of trying to preserve a snowflake. It's difficult, usually messy, and often ineffective. What I didn't realize is that I really shouldn't have any snowflakes - that what I was really after was a way to consistently reproduce, not a system to save the only copy the world would ever see. Of course configuration management already existed, why I didn't understand the problem and the solution that was already out there I don't know. Perhaps I was a latent server-hugger, or feared obsolence by a few lines of ruby - but configuration management (coupled with automated provisioning) meant I didn't need portability - didn't even want portability.

You see we really are at a point where a virtual machine is a commodity. Yes hypervisors and cloud platforms aren't quite at commodity level, many of them do things that are still quite novel and interesting, but at the lowest level, the VMs they offer up certainly are commodity. And as a commodity (a largely renewable commodity at that) it's easiest just to harvest them and redeploy. I'd even go so far as to say it's preferable to redeploy. If my 'known good' state is in configuration management, destroying the machine and redeploying it is trivial, I can confirm that the machine is in the correct state, that no changes outside of my configuration management scope are the cause of the problems. Because I can use the same definition of 'Just enough operating system' (JEOS) in all of my deployments (regardless of whether they are bare metal, VMware, AWS, or any other of a number of platforms, I know I can start with the same base, and then use configuration management to render my mass-produced server instance perfectly - regardless of environment.

There are other problems with portability (bandwidth constraints among them) but I am convinced that as sysadmins, we no longer want or need portability. We want replicability, the server instantation factory. If we revert back to collecting snowflakes, we'll spend all of our time trying to preserve them, and inevitably end up failing along the way.

[1] Black magic for most - I know that Euca says they can move AMIs between AWS and Euca and that it works for most of the images out there.

Rate this blog entry:
David Nalley is currently employed by Citrix as the Community Manager for the CloudStack project. In addition he's a long time contributor to the Fedora Project, where among other things he is currently serving on the Fedora Project Board. He's also contributed to in various forms to Cobbler, Zenoss, Opengroupware.org, OLPC Math4, and Sahana. He is a frequent speaker at Free Software conferences around the nation, and writes for a number of technical and open source media publications including Linux Pro Magazine and OpenSource.com
  • Guest
    Anon Friday, 01 March 2013

    The title is great!

    While working at VeriSign, we received a set of paper "Snowflakes" for Christmas. The idea was to take pictures of the snowflakes in different places. A co-worker took a picture of the snowflake next to an iPod his wife got for Christmas from her company... they stopped showing off the pictures sent back by employees.

    On to your entry, I was a "server-hugger" up until about 5months ago when I gave in to the fact that almost all clients want to have some kind of cloud architecture and the physical servers are being logistically shoved under the datacenter floor tiles like cabling, power and cooling. We all know we need them and that they are in there, we just don't care that much any-more where they are, what we care for is that interface/api that will allow us to create more instances to satisfy our needs.
    Now I fight with management that does not understand exactly what you mention on this blog-post! I could coin the IT Title "Snoflake Manager". I'm having to manage a huge catalog of pre-configured AMIs for each server type we have and we have small document that says what instances are needed to stand up a new stack, FACEPALM...
    The issue has been mentioned already about what is the plan to migrate all these AMIs into a vCloud environment and that is exactly what i'm trying to avoid. I'm trying to bring configuration management tools into our infrastructure so we can begin to have replicability to a known working status regardless of the underlying hypervisor.

    Needless to say, I will be sending them this way for them to read this post. Great timing!

    Cheers,

Leave your comment

Guest Tuesday, 21 October 2014

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect