satis egitimisatis egitimitengda.pro

Open@Blog

Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Login
Posted by on in CloudStack Tips
  • Font size: Larger Smaller
  • Hits: 16128
  • Print
  • Report this post

Puppet and CloudStack


Efficient use of CloudStack really demands configuration management (among other things). I've been a puppet user for many years predating my involvement with CloudStack, and thus I have a slight bias for puppet.

Thus it thrilled me to no end when I saw folks like Jason Hancock doing work around automating configuration of CloudStack instances. Jason really knows Puppet, and even operates a new Hosted Puppetmaster startup. Jason showed this off a few times at both PuppetCamp LA 2012, and at the CloudStack Collaboration Conference in 2012.

It's awesome work, and you should take the time to watch both of his videos and check out his blog.

The gist of what he was presenting is configuring instance metadata within CloudStack at deployment, having the instance read that metadata and set it as a fact, and then using case statements to apply different roles to the instances.

And then there was a knife.....plugin


Next I learned that the good folks at Edmunds.com had written a CloudStack plugin for knife. That was exciting in and of itself, especially as a CloudStack person. It wasn't just knife, which is a command-line tool for Chef, another configuration management tool. knife is commonly used to provision machines, and the folks at Edmunds.com had baked in some extra awesomeness. They had the ability to define an application stack based on a JSON definition of what the stack looked like.

So one could define a Hadoop Cluster like this in JSON, complete with network and firewall configuration:

"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
  {
    "name": "zookeeper-a, zookeeper-b, zookeeper-c",
    "description": "Zookeeper nodes",
    "template": "rhel-5.6-base",
    "service": "small",
    "port_rules": "2181",
    "run_list": "role[cluster_a], role[zookeeper_server]",
    "actions": [
      { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
    ]
  },
  {
    "name": "hadoop-master",
    "description": "Hadoop master node",
    "template": "rhel-5.6-base",
    "service": "large",
    "networks": "app-net, storage-net",
    "port_rules": "50070, 50030, 60010",
    "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
  },
  {
    "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
    "description": "Hadoop worker nodes",
    "template": "rhel-5.6-base",
    "service": "medium",
    "port_rules": "50075, 50060, 60030",
    "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
    "actions": [
      { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
      { "http_request": "http://${hadoop-master}:50070/index.jsp" }
    ]
  }


And then deploying a Hadoop Cluster is as simple as:

knife cs stack create hadoop_cluster_a


As a CloudStack guy I thought this was awesome, complex applications were suddenly deployable with ease, this is exactly the kind of automation that CloudStack is supposed to enable.

JEALOUSY


As a puppet afficionado though, it made me a bit sad, nothing existed like this for folks using puppet, and I was jealous.

Then I went to FOSDEM


At FOSDEM some folks from a mapping company were talking about their use of CloudStack, and commented that they wanted CloudStack resource types. At first, I was confused. They then showed me some work by Ken Barber on OpenNebula resource and types, and I realized that this was exactly what would fill the role of knife-cloudstack's stack functionality. It also would in many ways, truly be infrastructure as code. It would not just be configuring the OS and software on a machine, but it would be configuring the underlying machine itself, and indeed many machines. You could literally have most of your infrastructure as code.

That was awesome, but it was for OpenNebula, and not CloudStack - I started bugging folks like Teyo Tyree and Dan Bode about this.

Puppetconf


Then I attended Puppetconf, and saw the inimitable Dan Bode showing a similar (albeit evolved) resource type for Google Compute Engine. I was even more envious at this point.

And then just after Thanksgiving - Dan Bode unveiled resources and types for Apache CloudStack.

Now you can use puppet not only to query CloudStack, but you can also define your infrastructure.

cloudstack_instance { 'foo1':
  ensure     => present,
  flavor     => 'M1.Medium',
  zone       => 'FMT-ACS-001',
  image      => 'CentOS 6.3(64-bit)',
  network    => 'davidsnetwork',
}


As with other resource types, you can define defaults:

Cloudstack_instance{
  image   => 'CentOS 6.3(64bit)',
  flavor  => 'M1.Medium',
  zone    => 'FMT-ACS-001',
  network => 'davidsnetwork',
  keypair => 'davidskeys',
}


Which makes defining instances much easier, and looking something like this:

cloudstack_instance { 'bar':
  ensure  => $::ensure,
  group   => 'role=db'
}



Of course, that's still a single machine, so if we want to have a multi-machine LAMP stack.

class my_web_stack {
  cloudstack_instance { 'httpd':
    ensure => present,
    group  => 'role=httpd',
  }

  cloudstack_instance { 'mysql':
    ensure => present,
    group  => 'role=mysql',
  }
}


Hopefully this leaves you as excited, as you can now puppetize ALL of your CloudStack instances.

 

Rate this blog entry:
David Nalley is currently employed by Citrix as the Community Manager for the CloudStack project. In addition he's a long time contributor to the Fedora Project, where among other things he is currently serving on the Fedora Project Board. He's also contributed to in various forms to Cobbler, Zenoss, Opengroupware.org, OLPC Math4, and Sahana. He is a frequent speaker at Free Software conferences around the nation, and writes for a number of technical and open source media publications including Linux Pro Magazine and OpenSource.com
  • No comments made yet. Be the first to submit a comment

Leave your comment

Guest Saturday, 19 April 2014

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect