May 2008

"You have a broad selection of open source projects to choose from...It's not easy to get the equations right--how strong is the community or how does it fit with us."

Bud Tribble, vice president of software technology at Apple

One of the questions always asked about open source is whether it's ready for the enterprise. But framing the question in that fashion blurs the issue. With over 100,000 open source products available for download at the click of a mouse, there is no blanket answer comprehensive enough to describe the entire universe of open source products.

The real question facing an enterprise is whether, based upon its unique requirements, a specific open source product will satisfy its needs. Far from being a vaguely existential question, this question is extremely pragmatic, completely localized, and, as we shall see, wholly capable of being answered.

This article, extracted from chapter four of "Succeeding with Open Source" presents the Open Source Maturity Model (OSMM). The OSMM is designed to enable organizations to evaluate open source products and understand whether a product can fulfill the organization's requirements.

OSMM Methodology

While many discussions of open source focus on software and its functionality, the OSMM recognizes that mainstream IT organizations have many requirements beyond a given product's code base: support, training, documentation, integration, and services. The OSMM evaluates a product along all these dimensions, assigns a maturity score to each product element, and generates a numeric score assessing the overall maturity of the product. A number without a context is less than useful, so the OSMM comes with recommended minimum maturity scores. These minimum maturity scores offer guidance as to what level of maturity should be present for a product to be considered for three different types of use: experimentation, pilot/departmental, and production. Naturally, these minimum maturity scores are only recommendations and may be adjusted according to the specific needs and capabilities of the organization.

Why do we need the OSMM? Haven't plenty of organizations implemented open source successfully without the OSMM? That's true, but overlooks the changing nature of the open source user base. Future open source users will require more complete, more mature products for their use. The OSMM is targeted toward the new breed of open source user.

Early Adopters and Pragmatists

In "Crossing the Chasm", Geoffrey Moore identified two main types of technology users: early adopters and pragmatists. Early adopters are comfortable using unfinished products, whereas pragmatists prefer to wait for the mature product. Up to now, open source software (OSS) has been the province of early adopters; today, however, pragmatists are seriously considering open source solutions.

Traditionally, technology vendors begin by selling to early adopters, who are satisfied with the rudimentary products startups deliver. When vendors decide to begin selling to pragmatists, they experience a rude awakening. Pragmatists expect a vendor to deliver a complete product bundle that includes service providers, robust support, thorough documentation, and so on. The product requirements of early adopters and pragmatists are radically different--different enough that Moore characterizes the distance between them as a chasm. Most vendors fail to successfully leap across this chasm. Open source seems like it would not face this problem; after all, the creators of the product are not focused on selling to any type of customer--early adopter or pragmatist--because the product is free. Customers make their own decision about whether to use a product, and never need to interact with a sales representative. This aspect of open source products overlooks one important fact: Even though no vendor is involved, it doesn't mean that pragmatists renounce their requirements. In the absence of a vendor, pragmatists often look elsewhere to procure a mature product.

In Moore's book, he noted that these distinct types of customers require very different products. Early adopters will accept immature products offering a competitive advantage. They are willing to forego access to sophisticated support, do not insist on high-quality training and documentation, and will even accept a lower quality product to achieve advantage. Consequently, early adopters are willing to work with small technology suppliers who are engineering-centric, short-staffed, and whose employees are "different", as long as the company provides advanced products. While early adopters are easier to work with, they represent only about 15% of any market; enough to get started on, but not enough for a vendor to prosper.

Pragmatists, by contrast, demand mature products. Mature products must be high quality and fully functional, but these factors are just the opening ante for pragmatists. To be accepted by them, products also must be accompanied by elements that make them easy to use and efficient to run. Mature products come with a training program, a sophisticated support operation, well-written documentation, and marketing materials that make it easy to compare the product to its competitors to understand how it fits into a computing infrastructure. Pragmatists requirements start with a particular piece of software, but they expect it to be bundled with a number of other product elements. Only when the entire bundle is available will pragmatists feel comfortable implementing a product. It's much more work to sell to pragmatists, but they represent a very lucrative 85% of the market share.

If you're an ambitious vendor, you'll have to deliver what this portion of the market demands. The world of open source, however, turns this process upside down. If you examine the over 100,000 open source products, it is clear that there are few, if any, open source providers that deliver a bundled product at the level of maturity pragmatic organizations require. The vast majority of open source products are freely available for download, with the expectation that the user organization will create the bundled product itself.

This highlights the open source mature product dilemma: a technology provider that, because of the economics of open source, cannot deliver a mature product, and a pragmatic technology consumer that requires a mature product to begin implementation.

The Open Source World

In the open source world, product elements are delivered by independent entities, with very little control exerted by the development organization. An organization that wishes to assess the maturity of a product must identify how each of the elements will be procured and the level of maturity of each element. This means, for example, that if an organization wishes to assess the maturity of an open source network monitoring product, it must identify where training can be found for the product and how good the training is.

Because of the nature of open source development, organizations selecting software cannot expect what they get when selecting commercial software: a sales rep to track down answers to every question, a sales engineer who will perform a demo and perhaps even prototype an application, and a support organization to answer questions after the product is installed and running. Determining the maturity of the product is something the organization will need to take on. Open source offers organizations much more control of their destiny; it also imposes much more responsibility for their product choices.

One way to look at this is to depict the process differently: Rather than procurement from a single provider, it is more akin to creating a coalition of providers to deliver the finished mature product. In the world of open source, selecting software is less like going to a Wal-Mart and more like being a construction general contractor. General contractors draw together independent entities like carpenters, plumbers, electricians, tile setters, and a large number of other contributors. Each member of the project performs his or her task under the guidance of the general contractor, who is responsible for selection and assessment of the people and for the quality of the overall product.

The task for open source users is to identify the necessary product elements, assess their maturity, and determine whether the complete product meets the necessary maturity level for the intended use. The challenge is to use a consistent assessment mechanism that ensures nothing is skipped and provides a formal set of assessment criteria.

The OSMM

The vast majority of the open source products available are probably not useful for an IT (information technology) organization. If even 1/10 of 1 percent of them are potential candidates for use, that represents a pool of more than 100 products that must be assessed for their maturity for a particular organization.

Without a formal methodology that implements a standardized analytical framework, organizations are limited in their ability to assess the maturity of a product. A framework also helps to identify the elements of a product that require improvement. Of course, lacking a way to formally assess products, organizations cannot compare open source products to determine which it should use. It is to address this challenge that Navica developed the OSMM. The OSMM assesses a product's maturity in three phases:

  1. Assess vital product elements for maturity and assign a maturity score.
  2. Define a weighting for each element based on the organization's requirements.
  3. Calculate the product's overall maturity score.

Phase 1: Assess Element Maturity

The first phase identifies key product elements and assesses the maturity level of each element. Key elements are those that are critical to implementing a product successfully:

  • product software
  • support
  • documentation
  • training
  • product integrations
  • professional services

Each element is assessed and assigned a score via a four-step process:

Step one: define requirements. The purpose of this step is to define the organization's requirements for a particular element. For example, if an organization wants to implement an open source web content cache, it must determine what functionality it requires in the software based on the organization's purpose: Is it attempting to reduce bandwidth load or response time, or does it have another purpose? As another example, if an organization is implementing an open source J2EE application server, its training requirements will be vastly different if it already has significant experience with a commercial application server than if it is beginning to use one for the first time. Defining the requirements for an element is a key step in assessing the usefulness of a product for a particular organization.

Step two: locate resources. Due to the loose coupling of product resources, locating resources for open source products is more complex than it is for comparable commercial products. There probably won't be an "approved partner" list for most products. Locating the resources for an element is more challenging, but there are a number of identification methods that can assist an organization in implementing OSS. As an example, product forums can be searched to locate a service provider that can supplement an organization's own personnel resources.

Step three: assess maturity. This is the key activity in determining the usefulness of a product element. Determining where the element lies on the maturity continuum--from nonexistent to production-ready--lets an organization determine how likely the product will satisfy its requirements.

Step four: assign maturity score. After the maturity assessment is complete, a maturity score between 0 and 10 is assigned to document how well the product element meets the organization's requirements. The score serves as a concrete output of step three: It documents the consensus of the organization. Assigning a score also compels the organization to crystallize its judgment.

Element scores are also helpful when comparing different products. It's easy to compare, say, the training maturity for two different open source content management systems in light of the organization's needs. This can become a decision tool for selecting one product or another based on the specific requirements of the organization.

Finally, the maturity score serves as an input into improving the element's maturity. If a product's overall maturity score is satisfactory, but one element's maturity score is low, the organization can choose to take steps to improve that element's maturity.

Phase 2: Assign Weighting Factors

The OSMM assigns a weighting to each element's maturity score, allowing each element to reflect its importance to the overall maturity of the product. For example, the heaviest weighting is typically assigned to the product software, whereas other elements have lower weighting factors to reflect the fact that they are less critical than the software itself in determining overall product maturity. The default weightings for the elements are shown in Table 1.

Table 1: Default OSMM Element Weightings

Software 4
Support 2
Documentation 1
Training 1
Integration 1
Professional services 1
Total: 10

The weighted score of each element is summed to provide an overall maturity score for the product.

Organizations might choose to adjust the default weighting factors based on their specific needs. For example, if an IT organization is stretched very thin in terms of personnel, it might plan to have an open source product implemented by a professional services firm. In that case, it might increase the weighting factor for professional services to 2 or even 3 to reflect the relative importance of professional services.

This allows the OSMM the flexibility to apply to every organization's situation. A product's maturity score will reflect the organization's specific needs and resources. The only requirement for adjusting the maturity weighting is that the element scores must sum to 10, since the final step of the OSMM is to create an overall maturity score that is normalized to a 100 point scale.

Phase 3: Calculate Overall Maturity Score

After each element has been assessed and assigned a weighting factor, the overall product maturity score is calculated. The element scores are summed to give an overall product maturity score on a scale of 1 to 100, where the highest possible maturity score is 100.

A blank template is downloadable from the Navica website. This site also provides blank worksheet templates that organizations can use as they work through assessing product elements.

Recommended OSMM Scores

Calculating a score and using it for a decision leaves out one of the most important factors in any decision: its purpose. A maturity score in an abstract consideration is meaningless; what is critical is the maturity score a product needs for a particular use.

The recommended minimum scores vary according to whether an organization considers itself an early adopter or a pragmatist. Pragmatic organizations are less willing to take risks with software products and therefore require higher maturity scores. In other words, there is an inverse relationship between risk tolerance and required maturity score. Depending on whether your organization is an early adopter or a pragmatist, you should adjust your minimum maturity scores to reflect your risk tolerance.

It must be emphasized, of course, that the recommended minimum scores are just that: recommendations. You might choose to use a product even though it fails to achieve the recommended minimum score for your purpose. In fact, you might decide that your organization would like to use different values for the minimum scores. The purpose of the recommendations is to provide a good starting point for determining how mature a product needs to be for a given purpose. If you feel a different value makes more sense for you, that's perfectly fine.It's more important that you perform a maturity assessment and determine what your minimum acceptable score is than to rigidly adhere to recommendations that might not reflect your needs.

Conclusion

Many people have observed that OSS is a disruptive technology. It's radically different modes of software creation and distribution promise to shake up the IT industry and cause a massive shift of power from vendors to users. Less often observed is the fact that disruptive technologies also shake up assumptions and working practices entirely appropriate to the previous environment but unworkable in the new one.

The comfortable assumptions about the roles of vendor and user that underpinned the commercial software world must be superseded by a recognition that, in the open source world, the shift of power to users is accompanied by a shift of responsibility. In the future, users will be responsible for creating the mature product bundle required for pragmatic organizations to use a technology.

The OSMM was developed to assist in that bundle creation effort, offering organizations the ability to assess the maturity level of open source products. The OSMM can be a powerful part of your open source toolkit. The next time someone in your organization questions whether a particular open source product is "production-ready", consider using the OSMM to answer the question definitively.

The whitepaper upon which this article is based, as well as the OSMM templates mentioned in the article are available for download from the Navica website.

Share this article:

Cite this article:

Rate This Content: 
No votes have been cast yet. Have your say!