As explained by Taleb in AntiFragile,”9 predictive and prescriptive processes stand a high chance of failure as the large number of rigid interdependencies mean that Black Swan events, which are by their nature unpredictable, are much more likely.
The “Agile Movement”10 is fundamentally an acknowledgement of, and response to, this issue; and focuses upon the organizational processes required to achieve agile development. A diverse spectrum of complementary lean and agile methodologies now exist; the two most popular are known as Scrum and Kanban.11
However, as is hopefully now apparent, the very same arguments apply to rigid monolithic software! Structural modularity is the foundation upon which application agility is realized, and the value of OSGi in enabling popular agile processes will be the subject of this section.
“Customers frequently change their minds”
At its core, Scrum is a pragmatic methodology that acknowledges that things change in unforeseen and unforeseeable ways. Scrum acknowledges the existence of requirement churn, and adopts an empirical12 approach to software delivery, accepting that the problem cannot be fully understood or even defined up front. Scrum’s focus is instead on maximizing the team’s ability to deliver quickly and respond to emerging requirements.
Scrum is an iterative and incremental process, with the “sprint” being the basic unit of development. Each sprint is a “time-boxed”13 effort, meaning that it is restricted to a specific duration. The duration is fixed in advance for each sprint and is normally between one week and one month. The tasks and estimated commitment for a sprint are identified in a planning meeting. A review, or retrospective, meeting follows the sprint to review progress and identify lessons for the next sprint.
During each sprint, the team creates finished portions of a product. The set of features that go into a sprint come from the product backlog, which is an ordered list of requirements.
Scrum attempts to encourage the creation of self-organizing teams, typically by co-location of all team members, and verbal communication between all team members.
It is hopefully self-evident that applying Scrum to a large monolithic code base is difficult. Conversely, as will be shown, Scrum concepts map well to a highly modular code base comprised of many self-describing, strongly isolated OSGi bundles.
Kanban originates from the Japanese word “signboard” and traces back to Toyota, the Japanese automobile manufacturer, in the late 1940s.14 Kanban encourages teams to have a shared understanding of work, workflow, process, and risk, enabling the team to build a shared comprehension of problems and suggest improvements, which can be agreed by consensus.
Kanban places an emphasis on “work in progress:”
- Work-In-Progress (WIP) should be limited at each step of a multi-stage workflow. Work items are “pulled” to the next stage only when there is sufficient capacity within the local WIP limit.
- The flow of work through each stage of the workflow is monitored, measured and reported. By actively managing flow, the positive or negative impact of continuous, incremental and evolutionary changes to a system can be evaluated.
Kanban encourages small, continuous, incremental and evolutionary changes.
Kanban concepts map naturally to a highly modular code base comprised of many self-describing, strongly isolated artifacts; specifically, the Kanban WIP process idea maps directly to the subset of OSGi bundles that are being actively worked upon. Hence, Kanban pull-based flow rates can be mapped to OSGi bundles’ change/release rates. And as the degree of structural modularity increases, as the OSGi bundles become more fine-grain, the Kanban pull -based flow rates naturally increase, with each smaller OSGi bundle spending correspondingly less time in a WIP state.
The Agile Maturity Model
Fashioned after the Capability Maturity Model,15 which allows organizations or projects to measure their improvements on a software development process, the Modularity Maturity Model16 is an attempt to describe how far along the modularity path an organization or project might be. As is now argued, this, in turn, will directly dictate the level of agility that might be reasonably expected.
Keeping in step with the Capability Maturity Model we refer to the following six levels.
Level 1: Ad Hoc – No formal modularity exists. Dependencies are unknown. Applications have no, or limited, structure. Agile processes are likely to fail as application code bases are monolithic and highly coupled. Testing is challenging as changes propagate unchecked, causing unintentional side effects. Governance and change management are costly and acknowledged to be high-level operational risks.
Level 2: Modules – Named modules are used with explicit versioning. Dependencies are expressed in terms of module identity (including version). Maven, Ivy and RPM are examples of modularity solutions where dependencies are managed by versioned identities. Artifact repositories are used; however, their value is compromised as the artifacts are not self-describing. Agile processes are possible and do deliver some business benefit. However, the ability to realize Continuous Integration (CI) is limited by ill-defined dependencies. Governance and change management are not addressed. Testing is still failure-prone. Indeed, the testing process is now the dominant bottleneck in the agile release process. Governance and change management remain costly and continue to be high-level operational risks.
Level 3: Modularity – Module dependencies are now expressed via contracts (i.e. capabilities and requirements). Dependency resolution becomes the basis of software construction. Dependencies are semantically versioned, enabling the impact of change to be communicated. By enforcing strong isolation and defining dependencies in terms of capabilities and requirements, modularity enables many small development teams to efficiently work independently and in parallel. The efficiency of Scrum and Kanban management processes correspondingly increases. Sprints are now associated with one or more well-defined structural entities; i.e. the development or refactoring of OSGi bundles. Semantic versioning enables the impact of refactoring to be contained and efficiently communicated across team boundaries. Via strong modularity and isolation, parallel teams can safely sprint on different structural areas of the same application. Strong isolation and semantic versioning enable efficient/robust unit testing. Governance and change management are now demonstrably much lower operational risks.
Level 4: Services – Services-based collaboration hides the construction details of services from the users of those services, so allowing clients to be decoupled from the implementations of the providers. Services lay the foundation for runtime loose coupling. The dynamic find and bind behaviors in the OSGi service model directly enable loose coupling by enabling the dynamic formation of composite applications. All local and distributed service dependencies are automatically managed. The change of perspective from code to OSGi µServices increases developer and business agility yet further: new business systems being rapidly composed from the appropriate set of pre-existing OSGi µServices.
Level 5: Devolution – Artifact ownership is devolved to modularity-aware repositories, which encourage collaboration and enable governance. Assets may be selected on their stated capabilities. Advantages include:
- Greater awareness of existing modules
- Reduced duplication and increased quality
- Collaboration and empowerment
- Quality and operational control
As software artifacts are described in terms of a coherent set of requirements and capabilities, developers can communicate changes (breaking and non-breaking) to third parties through the use of semantic versioning. Devolution allows development teams to rapidly find third-party artifacts that meet their requirements. From a business perspective, devolution enables significant flexibility with respect to how artifacts are created, allowing distributed parties to interact in a more effective and efficient manner. Artifacts may be produced by other teams within the same organization or consumed from external third parties. The Devolution stage promotes code re-use and increases the effectiveness of offshoring/near shoring or the use of third-party, OSS or crowd-sourced software components. This, in turn, directly leads to significant and sustained reductions in operational cost.
Level 6: Dynamism – This final level builds upon Modularity, Services and Devolution, and is the culmination of the organization’s modularity/agility journey.
Business applications are rapidly assembled from modular components. As strong structural modularity is enforced (i.e. isolation enforced by the OSGi bundle boundary), components may be efficiently and effectively created and maintained by a number of alternate providers (onshore, nearshore, offshore, OSS, third party).
- As each application is self-describing, even the most sophisticated of business systems is simple to understand, to maintain and to enhance.
- As semantic versioning is used, the impact of change is efficiently communicated to all interested parties, including governance and change control processes.
- Software fixes may be rapidly deployed into production.
- The capabilities of existing applications may be rapidly extended.
- As the dynamic assembly process is aware of the capabilities of the hosting runtime environment, application structure and behavior may automatically adapt to location, allowing transparent deployment and optimization for public cloud or traditional private data center environments.
An organization’s modularization migration strategy will be defined by the approach taken to traversing these modularity levels. Most organizations will have already moved from an initial Ad Hoc phase to Modules. Meanwhile organizations that value a high degree of agility will wish to reach the Dynamism endpoint as soon as possible.
Figure 8: Modularity Maturity Model
As shown, each organization may traverse from Modules to Dynamism via several paths, adapting migration strategy as necessary.
- To achieve maximum benefit as soon as possible, an organization may choose to move directly to Modularity by refactoring the existing code base into OSGi bundles. The benefits of Devolution and Services naturally follow. This is also the obvious strategy for new greenfield applications.
- For legacy applications, an alternative may be to pursue a Services first approach; first expressing coarse-grained software components as OSGi services; then driving code level modularity (i.e. OSGi bundles) on a service by service This approach may be easier to initiate within large organizations with extensive legacy environments.
- Finally, one might move first to limited Devolution by adopting OSGi metadata for existing artifacts. Adoption of requirements and capabilities, and the use of semantic versioning, will clarify the existing structure and impact of change to third parties. While structural modularity has not increased, the move to Devolution positions the organization for subsequent migration to the Modularity and Services
It should be noted that the ability to pursue multiple alternative options is in itself a key indicator of an increasingly agile environment!
|As the level of structural modularity increases through adoption of OSGi, both Scrum and Kanban processes become correspondingly more efficient and effective.|
9 AntiFragile: How to live in a World we don’t Understand – Nassim Taleb. ISBN-13: 978-1846141560
16 An initial version of a Modularity Maturity Model; this proposed by Dr. Graham Charters at the OSGi Community Event 2011. The version in this paper has been adapted to emphasize Agility aspects.