The problem with SDS under Cinder

It’s been a great Summit here in Atlanta, we’ve had one of the most exciting and energetic Summit’s that I’ve had the opportunity to participate in so far.  Now that we’ve cleared the Exhibit hall, the booths are gone and most of the marketing and sales folks are on a plane heading home, it’s time for the Developer Community to get to work.


On the Cinder side we have quite a bit of work to do and I expect some great brainstorming in the Design Sessions on Thursday and Friday.  One topic in particular that is on the agenda is “What is a Cinder Driver”.  This session in my opinion is perhaps the most critical session we’ll have this week and the decisions that come out of it will have a significant impact on the future of Cinder.


The reason for this session is to try and get some consensus around how to deal with the most recent trend that I’m seeing in the Cinder community.  The latest trend from a number of the larger storage vendors is to implement a Software Defined Storage “driver”.  The design here is to implement a generalized driver, that acts as an interface to an external Controller that implements an abstraction very similar to what Cinder is already intended to provide.  This model duplicates a significant amount of code that exists in Cinder.  The Controller handles scheduling, pooling and abstraction of whatever supported devices For those that aren’t familiar, there are a number of examples that can be found on the web, but in short if you’re familiar with Cinder, you’ll recognize it as almost a duplicate pretty quickly.


From the perspective of a Vendor that builds this Software Package and implements the driver in Cinder this has a number of advantages.  The primary advantage is of course they have a Software Defined Storage implementation that they can sell to customers.  In addition, from an OpenStack/Cinder perspective, this also provides an additional advantage.  These SDS implementations allow for a single driver implementation in Cinder to send requests to the SDS Controller as if it were a single backend device just like we do with a standard driver currently.  The difference however is that the SDS Controller then takes that command and routes it through it’s own scheduler and abstraction layer to any number of backend devices that are configured in it’s pool of back-end resources.  This means that a Vendor can implement multiple backend devices but only needs to submit and maintain a single Driver in OpenStack.


I have mixed views on this, I certainly appreciate and understand a desire to pool resources and abstract the to higher level, but given that this is the whole purpose of Cinder it seems odd to me.  It’s a somewhat strange and in my opinion less than optimal design, mainly because the only apparent benefit is for the Vendor that is selling the SDS Controller, or possibly for the Vendor whose product is supported by the SDS Controller.  The latter advantage is troubling, because I view it as nothing more than a short-cut.  It’s a fact that contributing a driver to Cinder requires significant effort.  That effort encompasses not only the development of the driver itself (frankly that’s the easy part), but more more importantly it requires significant effort in terms of support, testing and most of all community participation.  The required effort is actually by design, I believe that requiring this effort helps deliver a better end product, and acts as a sort of filter between providers of products that actually want to enhance and advance OpenStack/Cinder as opposed to just use it as a sales tool.


There are multiple Storage Vendors proposing this model and design to the Cinder project this week.  My initial reaction has been to “-2” such proposals and to “reject” proposed BluePrints submitted to LaunchPad.  There seems to be a good deal of debate on this topic however.  Some (particularly those that are proposing this model) obviously feel strongly that this is a great solution.  Others feel as though this is fine as long as they implement CI testing for every back-end they claim support for.  I don’t agree with either of these viewpoints.  I believe what makes the combination of Open Source and Vendor/Proprietary Software work, is when there’s a balance.  I define that balance as giving back at least an equal amount of improvement to the project as the benefit that you’re extracting from it.  Implementing your own abstraction is very much at odds with this philosophy.


Cinder and OpenStack are Open Source SOFTWARE projects.  I believe the key to the balance and relationship between community and Vendors in the Cinder project is in fact because there’s been a clear line between the Software and the Hardware (Storage Backend).  By having this clear separation we’ve fostered a unique and effective balance, Vendors can contribute open software to support their hardware products, which in turns also drives them to improve the overall project.  This model has worked extremely well over the last couple of years.  I’m very concerned that if you break this model and begin to merge not only proprietary hardware but also proprietary software in to the project, it causes significant damage to the Cinder project.  The result is that vendors can easily focus on their product, and ignore the core Cinder project by basicly just ignoring it.  There’s no longer any real incentive for them to contribute, and even worse, the consumer loses the freedom of choice and we create a tiered version of the Cinder software; a higher tier for those that pay for a vendors software, and a lower tier for those that utilize the Open Source version.
As PTL I’ll continue to vote that we don’t go down this path, and keep the current model we have in Cinder.  The beauty of Open Source however is that everybody has a voice and a vote, if the overwhelming opinion is that this new direction is the way to go, then so beit.  I do however believe this will prove to be damaging to OpenStack and in particular to Cinder.


12 thoughts on “The problem with SDS under Cinder

  1. Disclosure, EMCer here.

    John – thanks for your contributions to the community. I agree with your post – up to a point. If you are looking at the Openstack use case as the central use case (as many would), there is no need for an “storage abstraction under the storage abstraction”. Well – there is a small one (common model not only for Nova instance volumes via Cinder – but pooling for use in Swift in others at the same time).

    I think the argument is more that there is an ecosystem beyond Openstack, and some customers will want an abstraction that can apply across those ecosystems. I wrote a blog (triggered by other dialog, but made me think of your post here that I’ve been intending to comment on) on this topic here:

    In any case, thanks again – and for one, I’m pushing hard for more and more contributions to Openstack from EMC!

    • Thanks for the feedback Chad. Great points, and I think you hit the crux of the issue for me. I do think that VIPR is a super cool solution for a number of eco-systems outside of OpenStack, but for an OpenStack solution it’s kinda “meh…”. As far as contributions, Xyang has really stepped up to the plate on behalf of EMC and it’s been fantastic!

  2. Great discussion.
    I have to say, I find myself continually unsatisfied by the definitions of software-defined storage presented here and elsewhere.
    Most of them center around a programmatic exposition of pre-defined storage capabilities, e.g. here’s your gold, silver, bronze, etc. — what do you want? Maybe we’ve changed the interface, but we really haven’t changed the underlying model.
    Maybe it’s the fact I work for VMware, but we’re pushing for a far richer model: storage subsystems expose potential capabilities, applications expose a policy-based manifest of what they need, and the SDS layer dynamically composes a storage container and associated data services precisely aligned on application boundaries. Change the policy, the SDS layer either instructs the storage to adjust, or transparently moves the workload to a new location where the new conditions can be met.
    The difference might be described as fast food where everything is made far in advance vs. a sit-down restaurant where your meal is cooked to order.
    As I’m not all that familiar with the direction of Cinder and related projects, I’m not sure whether this is part of the broader thinking or not. From what I understand, Cinder does bring a lot to the table in terms of consistent access to underlying resources (a good thing) but seems entrenched in a static model.
    Time will tell whether people demand a more dynamic model (such as we use with compute) or not.

    Best regards!

    — Chuck

    • Hi Chuck,
      I’ve recently shyed away from using the term SDS mostly because it seems pretty controversial and it seems there’s no real widely accepted definition. As far as your comment regarding “changed the interface, but not changed the model” there’s some validity there. However (disclosure, I work for SolidFire Inc) there are Vendors like SolidFire that are creating new Storage Products that push these boundaries and change the game so to speak.

      From a Cinder perspective the main focus is in fact to abstract varying hardware backends into Pools, but in addition we are adding things to make storage consumption for flexible and more dynamic. The key principal of course is “self serve” and yes to a large degree we’re limited by what the ‘traditional’ backend devices we support offer. What’s cool however though is we do offer some customization and differentiation through concepts like Types and allowing a user to “re-type” a Volume. Additionally, we provide a mechanism to extend capabilities via API extensions. In this case Cinder provides a mechanism for any individual deploying an OpenStack based Cloud to add any customized API feature that they wish (of course it might involve quite a bit of work, but the capability is there).

      VMWare is doing similar cool things with features like V-Vols etc. The big difference here IMHO is OpenStack (in a general sense) is geared more towards providing a base platform/Operating System to build a Cloud on as opposed to just providing a “Product” such as VSphere.


  3. As I said at the summit and in various follow-ups since, I agree that these big SDS platforms have a definite and significant cost to the cinder project as a whole. I therefore expect more of a vendor trying to merge such things.

    In the specific case of EMC, they’ve agreed to step up to the plate and put effort into trail blazing our new CI requirements, and I have no hesitation at all hitting -2 until I see some serious CI progress on their side. They have the experience and resources and apparent willingness to put effort there – if they do, then great, the benefit that work provides acts as on offset to the downsides of merging support for their project. I expect to keep this attitude up with any other vendor who wants to merge similar code – the quid pro quo is that they push the cinder project as a whole ahead in a way that is positive for everybody. Small, focused drivers do this inherently – even more so if they are for an opensource system (Ceph for example), but I’m fine with making the ‘community effort’ explicit.

    Duncan Thomas (Cinder core)

  4. Pingback: The Problem SDS Under OpenStack Cinder Solves | Cloud Architect Musings

  5. Pingback: Engage in technical discussions keeping in mind the OpenStack promise][ stefano maffulli | ][ stefano maffulli

  6. Pingback: Technology Short Take #43 - - The weblog of an IT pro specializing in virtualization, networking, cloud, servers, & Macs

  7. Pingback: Vendor hijack – contributions from vendors to open source projects | Die wunderbare Welt von Isotopp

    • Hi,

      So in my opinion things like Ceph are the other end of the spectrum here (the good end). Ceph’s a great project and backend for Cinder; the key distinction in these posts for me has been the difference between duplicating the scheduling and abstraction functionalities that already exist in Cinder and sitting them underneath Cinder as a driver. In other words, when you have the exact same goal as Cinder what’s the point 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s