Why Openstack is Different from Other Open Source Projects

The Openstack project feels different from other open source projects to me. Let me try to explain.  Henrik Ingo did an excellent analysis of open source project size versus governance structure a few years ago.  Essentially, the nine largest most vibrant open source communities were anchored inside non-profit foundations. The tenth largest was an order of magnitude smaller and existed inside a corporation. Henrik was a good engineer and provided his data and listed his assumptions.  He was not suggesting that the growth was causal, simply that there was a strong correlation.  He rightly observed that it will be fascinating to see what it all means using Openstack as an example when he presented his findings Summer 2011. (He wrote an excellent follow-up post comparing cloud projects later.)

Henrik’s analysis and observations presumes certain criteria about how the open source licensed project must be functioning in and of itself.  It has to be a “well run project” before getting to the foundation amplifier. Lots of people have described essentials of successful communities.  Some of my favorites include Dave Neary, Donny Berholz, and Kohsuke Kawaguchi, and of course there’s Jono Bacon’s essential reference.  I tried to capture and collect together the patterns and practices of successful open source projects in a blog post a while ago. 

Paula Hunter and I offered rationale for the foundation amplification when we were executive and technical directors at the Outercurve Foundation. What might be the cause of the amplification?  We believe it comes from the one thing ALL foundations do for their stakeholders around their projects. Foundations exist at the behest of their stakeholders to make clear the IP management requirements and provide IP risk mitigation. The project community must be well run from engineering and governance perspectives, but once a foundation exists, corporate participants have a clear on ramp for participation and the investment in the community can grow substantially. 

So Solid Engineering Practices + Strong Community Governance + Clear IP Management enables Growth. So far so good. 

But Openstack is somewhat unique as an open source project. It was 2010. Amazon had enormous early mover momentum in providing cloud services [2006]. Eucalyptus was open source-licensed but controlled by a single vendor (née 2009). Cloudstack was just published under an open source license [May 2010] but still closely held by Cloud.com. The time was ripe for a vendor neutral option.  Openstack was announced into existence by then Rackspace CEO, Lew Moorman, and NASA CTO, Chris Kemp, at OSCON 2010. 

A large collection of vendors jumped early and hard onto a simple infrastructure to evolve it rapidly forward into a marketplace of potential customers.  The project governance was created and people started gathering at summits. Openstack was correctly forced into a neutral, non-profit foundation within two years to clarify IP ownership. (Everyone had MySQL as an example on their mind, as it had been bought by Sun Microsystems [2008], and then bought and ransomed by Oracle [2010].)

But here’s where Openstack begins to break patterns. There was relatively little code at the start. It was created from the start to engineer forward.  Openstack is going through forced growth in a time frame that is a 20%-25% of the time of other large-scale successful open source-licensed infrastructure projects.  Instead of 20 years, the effort is becoming enormous in four, already driving serious vendor-lead products to market. 

Project	Started (age)	LoC Mid-point**	LoC Today	Contributors Mid-point	Contributors Today	Re-Arch	Foundation
Linux kernel	1991 (23)	4,000,000	17,000,000	186	1000	2002	2000
Apache (httpd)	1995 (19)	980,000	1,700,000	17	17	2002	1999
gcc	1987 (27)	1,150,000	6,900,000	50	94	1998	1987*
Openstack	2010 (4)	500,000	2,300,000	174	575	???	2012

[* gcc started in the Free Software Foundation; ** All data pulled from Ohloh.net, Wikipedia, and interviews]

Openstack didn’t evolve in simple use cases with the experimental and experiential use that other infrastructure open source-licensed projects saw (e.g. Linux, Apache) before large vendors got involved.  Openstack continues to demonstrate enormous growth and participation. As vendors begin to develop cloud delivery products and services out of the various Openstack projects, they are discovering holes in functionality requiring them rapidly to create new Openstack projects to fill the gaps.  Vendors are also starting to discover that Openstack itself may not scale to the needs of certain industry desires (e.g. Telco desires around NFV).  As well, all core infrastructure open source projects hit points in their organic histories where a reset was required in the architectural design to account for the stresses of new use and deployment.  Linux, Apache, and gcc were each re-architected at points in their histories to accommodate the organic growth of the project into new deployments and use.  

So interesting questions:

• When do the developers at the heart of Openstack projects collectively re-architect/re-factor/re-write core Openstack components to scale to the real world workloads that people are discovering customers need to manage over the next 5 years?
• How does the Openstack Foundation capture those consumer requirements in a fiercely competitive landscape of competitors?
• How will the Openstack Foundation adapt to the needs of its vendor, user, and consumer stakeholders? 

Open source communities are amazingly adaptable organisms.  It will be fascinating to watch Openstack as a community of projects and as a foundation continue to grow and evolve to meet the challenges of the cloud marketplace.