"Software assembly, whereby developers reuse already created code components, is dramatically changing how applications are built. The efficiencies are realized if development teams establish processes to manage how code is assembled, and to assure that the use of the code components is in compliance with the various licenses governing the code."
Doug Levin, CEO of Black Duck Software
Over the past ten years, the Internet and open source software (OSS) have enabled developers to fundamentally change the way they produce software. Increasingly, distributed teams are collaborating to assemble software from reusable components and their own proprietary code rather than building applications entirely from scratch.
The component-based development model is fundamentally changing the software industry. It enables organizations that develop software, either for commercial sale or for in-house use, to accelerate project timelines, improve software quality, and reduce development costs. If not managed properly, the complexity inherent in this new world of mixed-IP (Intellectual Property) can pose business and technical risks to an organization.
This paper draws on the experiences of the Black Duck Software team, our customers, and other industry experts to propose new approaches to managing IP in this new world. It describes a set of best practices that companies can use to avoid the risks and gain the benefits of the component-based approach to software development.
Development organizations have long understood the virtues of building new applications by re-using components already built and tested. Indeed, the evolution and rapid adoption of component-based architectures has been driven in part by their effectiveness in promoting economically-significant re-use. It is therefore appropriate to consider components as IP assets, optimize their usage, and protect their integrity. This applies equally to all components, whether they are internally developed or developed externally by a third party.
A development team intent on exploiting an externally developed commercial component does not typically purchase ownership of that component. Instead, they acquire a license to use that component in a specified manner--perhaps for only a certain number of developers working on a particular project, or with a specified royalty paid for each instance of a shipped product that includes the component. Thus, business judgment is required to ensure that the cost of licensing a commercial component is more than offset by the benefits.
Included in the cost analysis must be the effort required to ensure compliance with the license; for example, limiting the component's usage to a specific project, or tracking shipments in order to accurately calculate royalty payments.
The Open Source Impact
Over the past five years, OSS has risen to prominence, dramatically increasing the opportunity to re-use existing software. Developers can readily locate potentially useful components from a wide array of re-usable software components. Management Alternatives
Re-use can take many forms, including bundling independent components, integrating with or using libraries, and incorporating source code or source code fragments. In some cases these components can be modified as required to improve functionality, quality, performance, or footprint. As with commercial components, the ownership of externally developed OSS components and fragments remains with their authors. While most of these authors allow the commercial use of their software without initial payments or royalties, many have chosen to impose other constraints, such as:
- Usage reporting
- Publication of modifications and improvements
- License replication
- Resulting software must be open source
Such constraints are imposed by means of licenses, examples of which include the:
- Apache Software License (ASL)
- Common Public License (CPL)
- GNU General Public License (GPL)
- Mozilla Public License (MPL)
- New BSD License
A more complete listing of open source licenses is provided at http://www.opensource.org/licenses.
Open Source License Compliance
Development teams that incorporate OSS components or fragments of OSS components in their projects must comply with the terms of the licenses associated with those components. This can be challenging for several reasons including the following:
- There is wide variation in the obligations imposed by open source licenses, ranging from the BSD license (which has few obligations) to the GPL license (which has many)
- Some open source licenses are legally complex, introducing constraints whose business implications may not be obvious to a developer choosing to re-use a component
- The licenses of some commercial and open source components are mutually incompatible
- The origin of a particular source code fragment may be difficult to determine, effectively obscuring its license obligations
- Discovering the need to comply with a license late in a project's lifecycle can produce disagreeable tradeoffs, such as publishing all of the project's source code vs. increasing time-to-market by months while a component is replaced
In addition to the legal obligations imposed by licenses, developers who incorporate OSS components into an organization's projects may either be obligated by the terms of the license or feel a moral obligation to give something back to the community. The resulting actions may result in the inadvertent dilution or loss of the organization's IP.
Organizations can react to the challenges of OSS licenses in one of three ways. Some organizations turn a blind eye, ignoring the issue until a catalyzing event or crisis occurs. But the resulting misfortunes--major code rewrites, embarrassing negative publicity, delayed sales, failed acquisitions--make this an increasingly untenable approach. This is especially true in the new environment of increased business transparency, executive responsibility, and potential shareholder lawsuits.
Other organizations take the Draconian approach of banning all OSS re-use. This strategy is flawed because it:
- Is difficult to enforce
- Decreases productivity and agility compared to organizations that successfully re-use externally developed components
- De-motivates development teams by requiring them to apply scarce resources to develop compnents from scratch and test them rather than moving forward
The first two approaches fail to recognize the reality that open source and component reuse are here to stay. The third and recommended alternative is to encourage the re-use of both internally developed and externally developed (commercial and open source) components, while establishing controls at critical points in the project lifecycle, for example:
- When components are first added to a project
- When internally developed components are created or modified
- At every build
- At each phase transition in the development process
- When considering the contribution of a component to an open source project or the transfer of its ownership to another party
- Before acquiring a significant ownership interest in a software component
It is important to note that identifying problematic licensing issues early in the development cycle is tantamount to detecting serious software defects. The earlier a problem is detected, the less expensive it is to fix. While IP controls late in the development cycle, during testing or release assembly, are better than none, the earlier they happen in the lifecycle, the better.
Preparing for IP Management
To adopt best practices, several tasks should be considered by the individual or teams responsible for development and licensing. An organization intent on managing its software IP should identify the responsible business, legal, and technical individuals who will be involved in the process. The organization also should designate them as authorizers for each active project, and commission them as a group to oversee and manage the planning, implementation, and ongoing management of the process.
A good first step for the team is to define the boundary between internally developed and externally developed components. For example, a small organization that prefers taking a conservative approach may deem all of the code developed within its walls to be internal. Conversely, that company would view as external all software brought in from any outside source. Another organization might extend its view of internal software to include licensed proprietary software and software developed by its contractor and outsource partners. On the other hand, a department of a large corporation may want to consider its department internal and consider everyone else, including other departments in the same company to be external. Business judgment must be used to determine where the boundary should lie.
Another key preparation task is for the team to identify the development process phase transitions at which component re-use reviews will be conducted. The team should define criteria for designating an internally developed component as sensitive, for example due to trade secrets or patents, and develop and maintain a list of these sensitive items. The organization also should consider establishing and maintaining lists of:
- Licenses that are prohibited by the organization
- Externally developed components that, based on previous reviews, are approved for use in projects, and the situations in which use is approved
- Internally developed components that, based on previous reviews, are approved for contribution to open source projects or disposition to third parties
Once these lists have been created, the organization can use them to conduct an initial assessment of its existing code base(s). In this important preparatory step, the organization identifies and establishes the baseline pedigree, licenses, and components in use. As with any process improvement, an acceptable alternative approach involves introducing these steps incrementally over time.
Best Practices for IP Management
Whenever a development team considers adding new features or refining existing functionality in a project, it should explicitly seek internally developed and externally developed components that could accelerate delivery. The team should establish criteria for selecting and procuring these components. As would be expected, any component under consideration that fails to meet functionality, performance, reliability, maturity, or risk requirements should be eliminated.
The team should also eliminate any externally developed component whose license is on the prohibited license list, whose license obligations are financially or legally incompatible with the project's business objectives, or which uses other external components or fragments whose licenses are similarly unacceptable. For example, an organization developing a product that will be delivered under a proprietary license needs to be certain that any open source or proprietary licensed code that is incorporated can be safely included without causing an irreconcilable conflict between licenses. Any components that pass this initial test should be subjected to a make-buy analysis to determine whether or not its acquisition makes sense from a business perspective.
To protect the organization's critical IP, the creation and modification of all internally developed components should be tracked by recording a timestamp, the names of each author, and the applicable objectives and constraints. If a newly-created or modified component is suspected to be sensitive, the project's legal, business, and technical reviewers should be convened. If these reviewers deem the component to be sensitive, they should add it to the organization's list of sensitive internal components. By assessing sensitivity and license obligations at the point where a component is first being considered for re-use, decisions can be based on verifiable facts, eliminating last-minute surprises, guesswork, compromises, and risk-taking. This dramatically reduces the risk of schedule slippage, cost overruns, and damage to the organization's reputation. It also helps prevent the inappropriate re-use of critical IP.
For each component that a project's development team proposes to use within a project, the team should understand:
- The intended use and rationale for inclusion
- The component's sensitivity
- How the code will be incorporated
How the team deals with the component will depend, in part, on whether plans call for that component to be used temporarily or permanently. For example, the intent may be to use that component for a limited amount of time only to speed up prototyping or to advance the early phases of the development cycle, but not be intended to be made a permanent part of the code base.
Another determination the team should make is whether a component will be used only as is, or if modifications will be allowed, and if so, under which approvals. Development teams should describe the method of joining that will be used to incorporate the component into the project. This is an effective step because different types of joining can create different licensing obligations.
To achieve greater control over component re-use, teams should also take the following actions:
- Determine whether the component has been previously approved for the proposed form of use
- Declare the component's version and understand its license obligations as well as those of any externally developed components or fragments it contains or depends on
- Understand all potential incompatibilities between the component's license obligations and the license obligations of other externally developed components included in this project
- Present the above information to the project's legal, technical, and business authorizers and request approval to use the component as described
- If the component is internal and sensitive, the list that covers these items should be updated to note that component's inclusion in this project
- If the component is externally developed, its metadata and approval details should be recorded in the list of approved external components
Inspecting the code base on a regular basis decreases the likelihood that unexpected components will be introduced without being noticed. Therefore, at the creation of each project build or at release assembly, the development team should verify that:
- No unapproved sensitive internally or externally developed components or fragments have been added to the project
- No unapproved changes have been made to sensitive internally developed components
- No changes have been made to externally developed components whose form of use precludes changes, or requires that all changes be approved
Any inappropriate additions or changes discovered during verification should be immediately addressed, either by obtaining approval from the project's legal, technical, and business reviewers, or by backing out the offending modification. The root cause of any component misuse should be identified and corrected to ensure no subsequent regression. By promptly and diligently assessing every build and release, the development team will be able to detect errors when they are least expensive to correct.
At the completion of each build or release, the key metadata for all externally developed components should be recorded in the associated bill-of-materials. This enables demonstrable compliance with license obligations, and eliminates any uncertainty caused by changes between project releases by providing a clear audit trail. As a project completes a major development process phase, its legal, technical, and business reviewers should do the following:
- Verify that no unapproved sensitive internal or external components or fragments are used in the project
- Verify that no unapproved changes were made to sensitive internal components, and that no unapproved or precluded changes were made to external components
- Review the license obligations of all external components used in the project, and ensure compliance with these obligations
These phase reviews backstop the development team, and keep the legal, technical, and business reviewers engaged in the management of software re-use. They also verify that changes in the project's objectives have not created legal, technical, or financial inconsistencies with the licenses of components used in the project.
The rationale for contributing components to an open source project is beyond the scope of this report, as are the considerations involved in transferring ownership to a third party or creating a new open source project. However, if a contribution or transfer of a candidate component or fragment is deemed appropriate, the project's legal and business reviewers should:
- Determine whether the candidate component's sensitivity is an impediment to contribution or transfer
- Verify the right to contribute or transfer every externally developed component or fragment contained within the candidate
This helps to ensure that the organization does not inadvertently contribute code that shouldn't be added because of its sensitivity or because the organization is not entitled to supply it. If an organization is considering an acquisition that would include a significant interest in one or more software components, the designated set of legal, technical, and business reviewers should be charged with the following:
- Identifying all included components not owned by the supplier or target
- Assessing their license obligations with respect to the acquirer's compliance, business objectives, and legal policies
- Assessing the impact of any required rework or change on cost, revenue, and quality
Note that this best practice applies to a variety of situations in which financial investments are involved. Such situations include: company mergers and acquisitions, product acquisitions, joint venture formations, and venture capital investments.
By integrating these practices in to development processes, organizations will have far greater assurances of compliance with all relevant license obligations and far more effective protection of software IP. Adopting these practices will enable companies to be more aggressive in their use of the software assembly approach. That, in turn, will enable those companies to more quickly gain the benefits and competitive advantage this new development approach promises including accelerated project timelines, improved software quality, and reduced development costs.