Changes at OpenStack

OpenStack has become boring. Boring is good in enterprise. It's a sign of maturity, stability, and consistent growth. OpenStack is showing all of that. However, it has also had its share of problems. With great adoption, comes great bloat. As OpenStack started getting deployed in new use cases, it started to see new projects being created by those users to address their own niche.

To maintain quality, the OpenStack project came out with an integrated release model, where new projects had to go through a vigorous incubation process. Once matured, they had to go through a voting process to become part of the integrated release. It didn't work out well, as many projects failed to meet the standards set by the OpenStack project. It was a dead end.

OpenStack tried to solve that problem in 2015 by moving away from an integrated release model to a Big Tent model. Under the Big Tent model, community members were free to work on their projects without having to worry about going through the incubating and voting process.

The Big Tent model allowed projects to work independently of OpenStack. It was a self-service model, where OpenStack stopped managing those projects and started offering refined processes and tools so that communities could manage their own projects. These projects also had an option to coordinate their release with the six-month release cycle for OpenStack. It was seen as a better solution to the integrated release.

But the Big Tent model brought its own set of problems, as the ever-growing community started building more services around OpenStack, instead of focusing on the core of the project. It also lead to confusion. It was not clear anymore what was part of OpenStack and what was not. Many projects were started but then left unmaintained. It became a bigger problem than the integrated release.

Mark Shuttleworth, CEO of Canonical, criticized the Big Tent model and said OpenStack must move away from "BS as a Service" and focus on the core strength that's compute, storage, and networking.

Mirantis cofounder Boris Renski agreed with that assessment and told me in an interview that Mirantis itself is withdrawing from such projects and is going to focus on adding value to the OpenStack ecosystem. Mirantis ended up letting go of many people who were working on such projects.

The OpenStack Foundation was paying close attention to the changing dynamics. OpenStack takes a lot of beating for it, but people fail to see that there has never been such a massive open source project that enjoyed such growth. Making mistakes, learning from them, and fixing them is the natural course of action. That's exactly what OpenStack is doing.

Early this year at OpenStack Summit, Boston, the OpenStack Foundation gave clear signals of moving away from the Big Tent model. A lot of work is happening at the code level, community level, structuring level, and communication level.

"We are basically getting rid of the Big Tent name and doing some restructuring of the project," said Lauren Sell, vice president of marketing and community services for the OpenStack Foundation. She told us that they will have large board meetings in September and October to discuss this in more detail. We are expecting announcements at the upcoming OpenStack Summit.

So what would the new model look like?

Talk Is Easy; Show Me the Code

At the previous Summit, Mark Collier, COO of the OpenStack Foundation, talked about OpenStack as a "composable" infrastructure that allows the use of a mix of tools and features depending on the use cases. That's where OpenStack is heading. It's becoming composable or modular.

The latest release of OpenStack, code-named Pike, brings this Open Source Infrastructure as a Service (IaaS) platform closer to that composability.

Jonathan Bryce, the executive director of the OpenStack Foundation, told us that some of that composable work has gone into the Pike release. Depending on use case, services like Cinder, Ironic, or Swift can be used on their own or in combination with other services. Bryce gave the example of Cinder, the block storage service for OpenStack, which can now be set up as a standalone service. It can be integrated with other projects like Docker and Kubernetes. Cinder now comes with a snapshot feature that allows for reverting to an older working version if something goes wrong. Users can extend storage volumes without having to shutdown the virtual machines (VMs).

What if a customer doesn't want to run VMs? Pike has improved integration between Cinder and Ironic (a service that provisions bare metal machines instead of VMs).

"If you want bare metal servers instead of VMs, you still need a way to image the operating system on them. It can now be done with Cinder," said Bryce. Users can have clouds with Cinder and Ironic, without any virtualization, and they can run containers or workloads like machine learning directly on it. Irrespective of the use case, composability of OpenStack now enables users to do what they need.

With this release, they have also improved the network integration, specifically around network segmentation. Users can run Ironic (bare metal provisioning) and Neutron (networking) to get full network segmentation between physical services, meaning multitenant applications run in the cloud (i.e., scalability).

This release brings scalability to the next level. The community has been working on evolving the way Nova (a core service that provisions Compute) environment is deployed. Users can start off with one cell, add additional cells as they need, and scale out the environment to many thousands of servers. It enables users to achieve horizontal scaling. It's a major feature that will continue to see further iterations into the next release of OpenStack, code-named Queen.

Another major work going on in OpenStack is toward containerization of OpenStack Control Plane. A lot of customers are taking these services, putting them in containers, and then managing those containers using tooling like Ansible and Kubernetes. Now a new project called OpenStack Helm that uses Kubernetes helps users manage these services as containers.

That's some of the higher level stuff that users will see in Pike. There is a lot of under-the-hood work, too. It might sound trivial, but OpenStack is upgrading from Python 2.x, which will reach the end of its life in 2020. What may look like a small upgrade is actually a pretty huge effort. The work has been going on for more than a year now, because OpenStack has a very large codebase, and they need to ensure that everything will function as expected. The upgrade will definitely bring new features and improvements. What's more important is that it will future-proof OpenStack as customers run a cloud for 10 years or longer.

What's Next?

Bryce said that the teams have been more focused on user experience, especially around managing a cloud and keeping it running for years. It reflects the change in dynamics within the OpenStack ecosystem. Companies are increasing focus on operations and services and looking for tools that are oriented around the life cycle of the OpenStack cloud. They are looking at environments they can operate, manage, upgrade, scale, and more.

Some of these new challenges and solutions will be showcased by the OpenStack community at the upcoming OpenStack Summit in Sydney. AT&T, for example, will demonstrate the set of tooling they are using to run hundreds of OpenStack clouds.

"That's what we like about the OpenStack community. We can have open discussions and put things out in the open and then the community responds and we continue to move forward," said Bryce.

To better serve this community, OpenStack has also adjusted its release cycle. Now a new version is released at least three months ahead of the flagship OpenStack Summit. The new release cycle gives developers enough time to discuss long-term goals at the Summit, which was not possible before.

OpenStack has come out with a new event called Forum, which takes place at the Summit. What's unique about Forum is that it brings together developers, vendors, and users who had been communicating on mailing lists into the same physical room. It creates a very dynamic environment for discussions and solving problems. Forum also brings together members of adjacent communities, like Ansible and Kubernetes, under the same roof.

"When you are at the Forum, you know that the next release is due in three months so you have three months of head start to look at pain points and actually have long term plans," said Bryce. Sell said that the Boston Forum was a huge success, and they are bringing it back to Sydney.

In a nutshell, the OpenStack bloat is gone, and it's becoming an "agile" and nimble IaaS platform. It's becoming boring again, but boring is good!