Head in the Clouds

Over the last couple weeks I have been using exoscale, a swiss public cloud provider based on CloudStack. They just launched after a beta testing phase that I had the chance to be part of. Their offering is primarily aimed at developers. The folks at Exoscale were kind enough to give me couple "gift cards" during the CloudStack Geneva meetup and I was able to get going. Together with PCExtreme, Leaseweb, and iKoula they are one more European public cloud provider in production with Apache CloudStack that I know of.

Their cloud is almost straightforward: two data centers in Geneva and Vernier, with hardware hosted by Equinix. They run Apache CloudStack 4.0.2, the latest release and use KVM hypervisors on Ubuntu based servers. One customization that they made and that I am aware of is that they patched CloudStack to output logs using logstash and use Kibana for visuzalization. They offer CentOS 6.4 and Ubuntu 12.04/13.04 64 bit templates with instance types from 512MB with 1 core to 32 GB with 8 cores. Their development and operations team is relatively small for such an offering but they are backed by the Veltigroup a leading IT provider in Switzerland, which gives them a 20 person team for support. Their developers are seasoned IT infrastructure enthusiasts who participate in the DevOps, openBSD, Clojure and Pallet community. The lead developer, Pierre-Yves Ritschard, formerly with paper.li, recently participated in DevOps Days Paris and has contributed a Clojure client to CloudStack: clostack. They are embracing open source, not only by using it, but also by contributing to the various communities that make up the foundation of Cloud services.

While CloudStack comes with a powerful and efficient Web UI, exoscale decided to create their own UI and integrate it with a ticketing, monitoring and billing system that they developed. It reinforces the fact the CloudStack API is extremely rich and that the default UI was actually designed as a proof of concept rather than something that all users should use. The UI will please developers by its simplicity and straigthforward ease of use. From talking to them, I know they they will soon open source the python client they developed to build the UI backend. Pierre Yves told me it resembled a little bit my toy UI that uses Flask and builds a REST wrapper on top of the CloudStack API. See a snapshot of an instance view below:

(This is the official release announcement text for the 4.0.2 release. You can also find it on the Apache CloudStack blog, but I wanted to make sure we spread the news far and wide. Congrats to the project on another successful release!) 

The Apache CloudStack project is pleased to announce the 4.0.2 release of the CloudStack Infrastructure-as-a-Service (IaaS) cloud orchestration platform. This is a minor release in the 4.0.0 branch, which contains fixes for 40 bugs.

Apache CloudStack is an integrated software platform that allows users to build a feature-rich IaaS. CloudStack includes an intuitive user interface and rich API for managing the compute, networking, accounting, and storage for private, hybrid, or public clouds. The project entered the Apache Incubator in April 2012, and graduated in March 2013.

The 4.0.2 release includes fixes for a number of issues, including two minor security vulnerabilities (CVE–2013–2756 and CVE–2013–2758), problems displaying storage statistics, a fix for the SSVM HTTP proxy, support for CentOS 6.4, and other fixes.

Downloads

The official source code releases can be downloaded from:

Welcome to the April 1 issue of the Apache CloudStack Weekly News. Don't worry, no foolishness in this issue – just a quick recap of the week's most important events.

As you recall, we officially announced that the CloudStack project was graduating from the incubator last week. Though there were no events quite of that magnitude this week, there was plenty of discussion of new features, a new Website design proposal, and Chiradeep Vittal has unveiled a new tool for testing and development called QuickCloud that will come in handy for many CloudStack contributors and users.

Read the rest on the Apache CloudStack blog »

The board voted last week, but the announcement came out today, so it’s officially official: Apache CloudStack is now a top-level project under the Apache Software Foundation (ASF).

In the grand scheme of things, this may not sound like a big deal – the ASF announces new TLPs pretty often, and Apache currently boasts more than 100 TLPs. But a fair number of projects don’t make it, and the amount of work required for a project to go from incubation to graduation isn’t trivial.

Code Maturity

Many folks confuse incubation status with the maturity level of the code. It’s really important to understand that there’s not a direct correlation between the status of the code and the status of the project in the incubator. CloudStack, for example, entered the incubator as a mature project with quite a bit of real-world adoption.

Even if a project enters the incubator with an immature codebase, there’s no guarantee of graduation simply because the code becomes mature enough for production use – graduation is predicated on a different set of criteria, and releases are only one factor.

It is true that a release is an important step to graduation, though. A community needs to be able to demonstrate that it can follow Apache procedures and muster up a release that satisfies the release requirements. However, the release requirements aren’t “is this production-ready code?”

CloudStack is now an Apache Top Level Project (TLP) at the Apache Software Foundation (ASF), the announcement just came out. What the incubation period has meant for CloudStack has nothing to do with code maturity. CloudStack was mature and used in enterprise settings before it entered incubation at the ASF. What incubation meant was that CloudStack evolved into an open source community, self-governed by the Apache Way: transparency, meritocracy, respect, non-affiliation and consensus in no particular order. The community has learned and demonstrated that it understands the principles and processes laid by the Apache Software Foundation and that it can now operate more autonomously.

Growing an open source community is challenging, folks who participate come from various backgrounds, may seldom meet and interact mostly via emails, social media, instant messaging. Participants come from all over the world, work in different time-zones and donate their time after (and sometime while) dealing with day jobs and family. Participants rally around a project that they deem interesting, sometime just to lend a hand for a few months, or sometimes because their day job requires them to do it. In that very heterogeneous and fluctuating mix, an open source software community emerges, self-governed, sustainable and non-affiliated. In the last 12 months CloudStack has done just that, building a community from the ground up, developing and understanding the principles laid out in our bylaws, adapting -if need be- people's way of developing software, getting to know each other, welcoming new members every day and setting the foundation for a sustainable software.

When growing a community it is fairly natural to want to measure how well we are doing and how healthy the community is. Over the last several months I have started collecting some data to analyze our community, trying to see how we were doing and interacting. I mostly looked at our public mailing lists doing a study similar to the one done about comparing CloudStack, OpenNebula, OpenStack and Ecualyptus. Secretly, this was also a good way for me to sharpen a few skills on BigData, not that big actually but I used MongoDB instead of MySQL so that qualifies as BigData :). Defining membership in an open source community is a challenge since there is no concept of membership, even the concept of contribution is ill-defined. What constitutes a contribution ? Which channels need to be considered ? In the case of ASF for instance, contribution to code may only mean being a committer, but a committer is someone with write access to the code. Just counting committers will leave out all the folks sending patches, doing testing, doing user support, translating documentation, giving talks and so on. Also while at the ASF everything happens on the mailing list, what about IRC channels, social media like Facebook, Twitter and Linkedin, and what about other communities that may arise around a particular software: sub-projects, user groups etc. In this analysis I decided to only look at our public mailing lists but there is more to it than just this data source.

The two figures below show the number of individual contributors measured by unique email addresses used to send messages to the users and developers mailing lists. The red lines represent data from the users and developers mailing list prior to entering incubation at the ASF. The blue lines represent the ASF specific lists. Significant is the impact that the move to the ASF has had on the number of contributors. The developers list has peaked over 200 per month and the users list has peaked over 150 per month so far (figure on the left). The last data point is March (as of March 21st) and numbers will go up by the end of the month. The graph on the right shows the accumulation of contributors, adding all unique email addresses every month into a set. This shows us again that the move to ASF has had a huge impact on the growth rate of the community and that both lists grow at relatively the same pace. Adding the accumulated number of contributors to both list and removing duplicates present in both sets, this gives us a magic number of 722 CloudStack contributors to date.