satis egitimisatis egitimitengda.pro

Open@Blog

Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Login
Subscribe to this list via RSS Blog posts tagged in configuration management

Building CloudStack RPMs with Ansible

Posted by on in Open Source
I've been hearing a lot about Ansible lately. First I've seen folks like Paul Angus building tooling around installing CloudStack with Ansible. Ansible intrigues me a bit; first it's largely being shepherded by Michael DeHaan, who originally wrote Cobbler and really eased a lot of pain for sysadmins needing to provision machines; so his work has some immediate credibility because of how awesome Cobbler was to use. Second, the entire decentralized config management angle is interesting. I like how minimalistic it is; and while I don't think that's necessarily a good fit for every environment, it is compelling for some. Finally, I see the blurring of lines between config management and workflow/job automation and that makes Ansible pretty versatile in my mind.   I tend to learn best when I have a concrete project to actually apply new tools to, so when the hosted puppet master service I had been using went permanently offline, I decided to recreate some of the tooling around building CloudStack RPMs in Ansible. I started out with a very basic playbook, which worked reasonable well.

Playbooks, in Ansible parlance, are a place to both define system configuration, as well allowing a known order for workflow automation. I first started with just defining a build environment for building CloudStack RPMs.

Then I had the chance to listen to Michael DeHaan showing off Ansible at a DevOps DC meetup in November, and one of his code snippets had an unknown-to-me built in variable that cut my playbook length in half (as well as the number of ssh calls as a result.)  Specifically I went from a playbook like:

  - name: Install wget
    yum: pkg=wget state=latest
  - name: Install rpm-build
    yum: pkg=rpm-build state=latest

to:



  - name: Install deps and niceties
    yum: pkg={{ list }} state=latest
    with_items:
     - wget
     - rpm-build
     ...

This made things much more efficient, and I pushed further from merely configuring the environment to also using Ansible as a bit of a workflow automation tool. So I added the entire build portion as well.

Things weren't 100% happy though - a number of the Maven dependency downloads failed, which caused compilation to fail. I really need to setup a Nexus mirror for CloudStack dependencies both to speed things up as well as to ensure they are reliably available for building. But this failure isn't the fault of Ansible, so can't really fault it here.

The end result is being able to spin up a fresh machine, point Ansible to it, applying the playbook and coming back (admittedly building the RPMs takes a long time) after a long cup of coffee and finding RPMs done. You can see my admittedly beginning attempts at this playbook here:
https://github.com/ke4qqq/ansible_cloudstack_rpmbuild

If you need to build CloudStack RPMs, or want to  see a very basic Ansible playbook you can look at:
https://github.com/ke4qqq/ansible_cloudstack_rpmbuild

Hits: 22206
Rate this blog entry:
Continue reading Comments

The LinuxTag Hack

Posted by on in CloudStack Tips

What do you do when you go to LinuxTag the premier Open Source conference in Berlin Germany ? You give a talk, you hand out tee-shirts at the CloudStack booth, you explain Cloud computing and you hack a CloudStack driver for SaltStack while patching libcloud.

The talk: Talking about Clouds is nice and all, but after many years and many talks, I shamelessly admit that it gets a little old. So lately I have been working on BigData, both as a backend to CloudStack (think Ceph, Riack CS, Gluster) and as a workload to a cloud. I am talking about using Apache Whirr or Apache Provisionr incubating to deploy "one-click" hadoop clusters on public clouds. It is a long story that I will keep for another post as I am trying to write this before going to bed. but check out the slides and keep an eye on pallet and exoscale.

The booth: An open source booth is ..well a booth. I came with my pop-up banner, table cloth, tee-shirts, post-cards, USB stick/bottle openers, it feels a little bit like a traveling sales man, not that I would know but I imagine it like this. I have to explain that the 2 and 3 XL shirts will shrink quite a bit and will fit perfectly people's M or L frames. Then I point at the banner to showcase the magnificient CloudStack UI, I explain that there is an API server behind it. Sometimes I launch devcloud and do a live demo to bring them to their knees, sometimes I have to ask for help on IRC to answer a question, and sometimes a german developer wants to trade a tee-shirt against illegal substances not to be named. Life in the fastlane let me tell you. But that's what it's like to build a community, it is very much an evangelization process.

...
Hits: 18197
Rate this blog entry:
Continue reading Comments

Puppet and CloudStack

Posted by on in CloudStack Tips


Efficient use of CloudStack really demands configuration management (among other things). I've been a puppet user for many years predating my involvement with CloudStack, and thus I have a slight bias for puppet.

Thus it thrilled me to no end when I saw folks like Jason Hancock doing work around automating configuration of CloudStack instances. Jason really knows Puppet, and even operates a new Hosted Puppetmaster startup. Jason showed this off a few times at both PuppetCamp LA 2012, and at the CloudStack Collaboration Conference in 2012.

It's awesome work, and you should take the time to watch both of his videos and check out his blog.

The gist of what he was presenting is configuring instance metadata within CloudStack at deployment, having the instance read that metadata and set it as a fact, and then using case statements to apply different roles to the instances.

And then there was a knife.....plugin


Next I learned that the good folks at Edmunds.com had written a CloudStack plugin for knife. That was exciting in and of itself, especially as a CloudStack person. It wasn't just knife, which is a command-line tool for Chef, another configuration management tool. knife is commonly used to provision machines, and the folks at Edmunds.com had baked in some extra awesomeness. They had the ability to define an application stack based on a JSON definition of what the stack looked like.

So one could define a Hadoop Cluster like this in JSON, complete with network and firewall configuration:

"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
  {
    "name": "zookeeper-a, zookeeper-b, zookeeper-c",
    "description": "Zookeeper nodes",
    "template": "rhel-5.6-base",
    "service": "small",
    "port_rules": "2181",
    "run_list": "role[cluster_a], role[zookeeper_server]",
    "actions": [
      { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
    ]
  },
  {
    "name": "hadoop-master",
    "description": "Hadoop master node",
    "template": "rhel-5.6-base",
    "service": "large",
    "networks": "app-net, storage-net",
    "port_rules": "50070, 50030, 60010",
    "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
  },
  {
    "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
    "description": "Hadoop worker nodes",
    "template": "rhel-5.6-base",
    "service": "medium",
    "port_rules": "50075, 50060, 60030",
    "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
    "actions": [
      { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
      { "http_request": "http://${hadoop-master}:50070/index.jsp" }
    ]
  }


And then deploying a Hadoop Cluster is as simple as:

knife cs stack create hadoop_cluster_a


As a CloudStack guy I thought this was awesome, complex applications were suddenly deployable with ease, this is exactly the kind of automation that CloudStack is supposed to enable.

JEALOUSY


As a puppet afficionado though, it made me a bit sad, nothing existed like this for folks using puppet, and I was jealous.

...
Hits: 20865
Rate this blog entry:
Continue reading Comments

Puppet Labs: The Leading Open Source Data Center Automation Solution

One of the things we have found about running clouds at scale is the need for automation. To that end we often conduct training events called Build a Cloud Days with our open source partners. We like PuppetLabs puppet for automating configuration of CloudStack as well as Zenoss for automating the discover of cloud infrastructure to bring them under monitoring.  

One of the best ways to get up to speed on using puppet is to attend a Puppet Camp. The next one will be hosted in Atlanta on February 3rd.  

"Puppet Camp is a community oriented gathering of Puppet users and developers. You’ll have the opportunity to network with a diverse group of Puppet users, benefit from insightful lectures delivered by prominent community members, and be able to share experiences and discuss potential implementations of Puppet during our attendee generated breakout sessions."

If you can't attend this session there are many other Puppet Camps being held worldwide listed on the PuppetLabs website

Hits: 10067
Rate this blog entry:
Continue reading Comments

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect