satis egitimisatis egitimitengda.pro

Open@Blog

Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Login
Subscribe to this list via RSS Blog posts tagged in Apache

I had a recent discussion with some folks wondering why there was now an option for 32 or 64-bit System VMs with CloudStack 4.3. I provided an answer, and linked back to some mailing list discussions. I figured this might be of general interest, so I’d document in the short term with a blog post.

For background, system VMs provide services like dealing with snapshots and image templates, providing network services like load balancing, or proxying console access to virtual machines. They’ve historically been 32-bit. The reason for this is that the 32-bit arch has been very efficient with memory usage, and since these are horizontally scalable it’s easy to just spin up another.

But you can have either – which do you pick?

Depending on the workload you might have a different answer. Some hypervisors work better with one arch over the other; and that might be a factor; but ignoring hypervisors lets examine the reason you’d want to use either. 32-bit: 32-bit operating systems are pretty efficient with their use of memory compared to 64-bit. (e.g. the same information typically occupies less space in memory). However there are limits on memory. (Yes, you could use PAE with a 32-bit kernel to get more addressable memory, but there is considerable CPU overhead to do so – which makes it inefficient given that all of this is virtualized) The 32-bit kernels also have a limit on how much memory is used by the kernel. This is really where the use case of 64-bit System VMs evolved from. Because one of the system VM functions is providing load balancing, the conntrack kernel module had a practical limit of ~2.5M connections – and that left precious little room for the kernel to do other things. CloudStack orchestrates HAProxy as the default virtual LB, which in turn uses conntrack. Having a heavily trafficked web property behind CloudStack’s 32-bit virtual load balancer might run into that limitation.

64-bit: Not nearly as efficient with memory usage; however it can address more of it. You’ll actually tend to need more memory for the same level of functionality; but if you need to push the envelope further than a 32-bit machine, then at least you have an option to do so.

...
Hits: 2859
Rate this blog entry:
0
Continue reading Comments

ApacheCon approaches

Posted by on in Open Source

It doesn't seem possible, ApacheCon is less than a week away. CloudStack Collaboration Conference follows shortly. I am excited about seeing ApacheCon this year, the schedule is huge and contains a ton of interesting content. That actually may be a detriment - so much content it will make deciding what talks to see difficult.

Particularly interesting to me is the ton of big data talk. There are plenty of big data projects at the ASF, and those projects have managed to bring 5 days worth of big data content to ApacheCon, coupled with 3 days worth of Lucene/Solr content. Keeping in mind that the ASF is the home for the majority of open source big data projects and ApacheCon becomes a must-attend event if you care about big data.

But ApacheCon is more than just big data, there are tracks for cloud and mobile development, as well as perennial favorites like Traffic Server, and Tomcat. 28 tracks in total.

All of this content is great; and I look forward to learning a lot while I am at ApacheCon, but it ignores the most valuable reason that I am attending: the hallway track. Being able to converse with  many members of various Apache project communities is invaluable.

I hope to see you there.

Hits: 2313
Rate this blog entry:
0
Continue reading Comments

Building CloudStack RPMs with Ansible

Posted by on in Open Source
I've been hearing a lot about Ansible lately. First I've seen folks like Paul Angus building tooling around installing CloudStack with Ansible. Ansible intrigues me a bit; first it's largely being shepherded by Michael DeHaan, who originally wrote Cobbler and really eased a lot of pain for sysadmins needing to provision machines; so his work has some immediate credibility because of how awesome Cobbler was to use. Second, the entire decentralized config management angle is interesting. I like how minimalistic it is; and while I don't think that's necessarily a good fit for every environment, it is compelling for some. Finally, I see the blurring of lines between config management and workflow/job automation and that makes Ansible pretty versatile in my mind.   I tend to learn best when I have a concrete project to actually apply new tools to, so when the hosted puppet master service I had been using went permanently offline, I decided to recreate some of the tooling around building CloudStack RPMs in Ansible. I started out with a very basic playbook, which worked reasonable well.

Playbooks, in Ansible parlance, are a place to both define system configuration, as well allowing a known order for workflow automation. I first started with just defining a build environment for building CloudStack RPMs.

Then I had the chance to listen to Michael DeHaan showing off Ansible at a DevOps DC meetup in November, and one of his code snippets had an unknown-to-me built in variable that cut my playbook length in half (as well as the number of ssh calls as a result.)  Specifically I went from a playbook like:

  - name: Install wget
    yum: pkg=wget state=latest
  - name: Install rpm-build
    yum: pkg=rpm-build state=latest

to:



  - name: Install deps and niceties
    yum: pkg={{ list }} state=latest
    with_items:
     - wget
     - rpm-build
     ...

This made things much more efficient, and I pushed further from merely configuring the environment to also using Ansible as a bit of a workflow automation tool. So I added the entire build portion as well.

Things weren't 100% happy though - a number of the Maven dependency downloads failed, which caused compilation to fail. I really need to setup a Nexus mirror for CloudStack dependencies both to speed things up as well as to ensure they are reliably available for building. But this failure isn't the fault of Ansible, so can't really fault it here.

The end result is being able to spin up a fresh machine, point Ansible to it, applying the playbook and coming back (admittedly building the RPMs takes a long time) after a long cup of coffee and finding RPMs done. You can see my admittedly beginning attempts at this playbook here:
https://github.com/ke4qqq/ansible_cloudstack_rpmbuild

If you need to build CloudStack RPMs, or want to  see a very basic Ansible playbook you can look at:
https://github.com/ke4qqq/ansible_cloudstack_rpmbuild

Hits: 15691
Rate this blog entry:
Continue reading Comments

b2ap3_thumbnail_cloudstackcollab.png

Are you looking to run your own cloud computing environment and want to interface with other users? Are you trying to cut through the hype and find out how to really run a cloud? Are you looking for a great personal conference that is run by users not marketing dollars? Then you should be in Amsterdam next week at the CloudStack Collaboration Conference. It's a user run confererence designed to bring the users, developers and other supporters of Apache CloudStack to collaborate on the development and best practices for running cloud computing. 

Just One Word - Plastic DevOps

As the cloud computing has accelerated the time to deployment for infrastructure the need to keep up with our capibilities is leading to a culture change, DevOps. The methodologies and practices used to fully utilize cloud computing are being championed by the elite operators of Cloud Computing infrastructure. The CloudStack Collaboration Conference has a full program to help provide some of the best thinking on devops and cloud all in one place. 

Patrick Debois the man who coined the term DevOps and conducts DevOps Days worldwide will be the opening keynote at CloudStack Collaboration conference. 

Dell's Chief DevOps Evangelist and the co-host of the popular DevOps Cafe podcast will be talking about how the principles of DevOps combined with the emerging trend in Software Defined Networking(SDN) are driving the cloud. 

...
Hits: 23928
Rate this blog entry:
0
Continue reading Comments

Why I will go to CCC13 in Amsterdam ?

Posted by on in Cloud News

Aside from the fact that I work full-time on Apache CloudStack, that I am on the organizing committee and that my boss would kill me if I did not go to the CloudStack Collaboration conference, there are many great reasons why I want to go as an open source enthusiast, here is why:

It's Amsterdam and we are going to have a blast (the city of Amsterdam is even sponsoring the event). The venue -Beurs Van Berlage- is terrific, this is the same venue where the Hadoop summit is held and where the AWS Benelux Summit was couple weeks ago. We are going to have a 24/7 Developer room (thanks to CloudSoft) where we can meet to hack on CloudStack and its ecosystem, three parallel tracks in other rooms and great evening events. The event is made possible by the amazing local support from the team at Schuberg Philis, a company that has devops in its vein and organized DevOps days Amsterdam. I am not being very subtle in acknowledging our sponsors here, but hey, without them this would not be possible.

On the first day (November 20th) is the Hackathon sponsored by exoscale. In parallel to the hackathon, new users of CloudStack will be able to attend a full day bootcamp run by the super competent guys from Shapeblue, they also play guitar and drink beers so make sure to hang out with them :). Even as cool is that the CloudStack community recognizes that building a Cloud takes many components, so we will have a jenkins workshop and an elasticsearch workshop. I am big fan of elasticsearch, not only for keeping your infrastructure logs but also for other types of data. I actually store all CloudStack emails in an elasticsearch cluster. Jenkins of course is at the heart of everyone's continuous integration systems these days. Seeing those two workshops, it will be no surprise to see a DevOps track the next two days.

Kicking off the second day -first day of talks- we will have a keynote by Patrick Debois the jedi master of DevOps. We will then break up into a user track, a developer track, a commercial track and for this day only a devops track with a 'culture' flavor. The hard work will begin: choosing which talk to attend. I am not going to go through every talk, we received a lot of great submissions and choosing was hard. New CloudStack users or people looking into using CloudStack will gain a lot from the case studies being presented in the user track while the developers will get a deep dive into the advanced networking features of CloudStack including SDN support -right off the bat-. In the afternoon, the case studies will continue in the user track including a talk from NTT about how they built an AWS compatible cloud. I will have to head to the developer track for a session on 'interfaces' with a talk on jclouds, a new GCE interface that I worked on and my own talk on Apache libcloud for which I worked a lot on the CloudStack driver. The DevOps track will have an entertaining talk by Michael Ducy from Opscode, some real world experiences by John Turner and Noel King from Paddy Power and the VP of engineering for Citrix CloudPlatform will lead an interactive session on how to best work with the open source community of Apache CloudStack.

After recovering from the nights events, we will head into the second day with another entertaining keynote by John Willis. Here the choice will be hard between the storage session in the commercial track and the 'Future of CloudStack' session in the developer track. With talks from NetApp and SolidFire who have each developed a plugin in CloudStack plus our own Wido Den Hollander (PMC member) who wrote the Ceph integration the storage session will rock, but the 'Future of CloudStack' session will be key for developers, talking about frameworks, integration testing, system VMs...After lunch the user track will feature several intro to networking talks. Networking is the most difficult concept to grasp in clouds (IMHO). The storage session will continue with a talk by Basho on RiakCS (also integrated in CloudStack) and a panel. The dev track will be dedicated to discussions on PaaS, not to be missed if you ask me, as PaaS is the next step in Clouds. To wrap things up, I will have to decide between a session on metering/billing, a discussion on hypervisor choice and support, and a presentation on the CloudStack community in Japan after Ruv Cohen talking about trading cloud commodities.

...
Hits: 10775
Rate this blog entry:
0
Continue reading Comments

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect