Table of Contents
Foreword xviiPreface xxiAcknowledgments xxviiAbout the Author xxixPart I: Overview of Software Agility 1Chapter 1: Introduction to Agile Methods 5
Achieving Competitive Advantage in a Software Economy 5
Enter Agile Methods 6
Agile at Scale 7
A Look at the Methods 8
The Trend to Agile Adoption 10
Business Benefits of Software Agility 11
A Brief Look at XP, Scrum, and RUP 13
Summary 15Chapter 2: Why the Waterfall Model Doesn’t Work 17
Problems with the Model 19
Assumptions Underlying the Model 20
Enter Corrective Actions via Agile Methods 26Chapter 3: The Essence of XP 29
What Is XP? 29
What’s So Controversial about XP? 30
What’s So Extreme about XP? 30
The Fundamental Tenet of XP 31
The Values, Principles, and Practices of XP 33
The Process Model for XP 38
Applicability of the Method 39
Suggested Reading 40Chapter 4: The Essence of Scrum 41
What Is Scrum? 41
The Roles in Scrum 42
The Philosophical Roots of Scrum 42
The Values, Principles, and Practices of Scrum 43
Key Practices of Scrum 44
The Fundamental Tenet of Scrum: Empirical Process Control 45
The Process Model for Scrum 46
On Scrum and Organizational Change 48
Applicability of the Method 48
Suggested Reading 49Chapter 5: The Essence of RUP 51
What Is RUP? 51
Key Characteristics of RUP 51
Roots of RUP 52
Agile RUP Variants 60
Applicability of the Method 61
Suggested Reading 62Chapter 6: Lean Software, DSDM, and FDD 63
Lean Software Development 63
Dynamic Systems Development Method 65
Feature-Driven Development 70Chapter 7: The Essence of Agile 75
What Are We Changingwith Agile? 75
The Heartbeat of Agile: Working Code in a Short Time Box 81
Summary 85Chapter 8: The Challenge of Scaling Agile 87
Apparent Impediments of the Methods 88
Impediments of the Enterprise 90
Summary 94Part II: Seven Agile Team Practices That Scale 95Chapter 9: The Define/Build/Test Component Team 101
What Is the Define/Build/Test Component Team? 102
Eliminating the Functional Silos 104
The Roles and Responsibilities of an Agile Component Team 106
Creating Self-Organizing, Self-Managing Define/Build/Test Teams 109
Distributed Teams 114Chapter 10: Two Levels of Planning and Tracking 115
A Generalized Agile Framework 116
Summary: Two Levels of Planning 120Chapter 11: Mastering the Iteration 123
Iteration: The Heartbeat of Agility 123
The Standard, Two-Week Iteration? 123
Planning and Executing the Iteration 124
Iteration Planning 125
Iteration Execution 129
Iteration Tracking and Adjusting 132
Iteration Cadence Calendar 135Chapter 12: Smaller, More Frequent Releases 139
Benefits of Small Releases 139
Defining and Scheduling the Release 141
Planning the Release 144
Release Tracking 147
The Release Roadmap 149
Agile at Scale Preview: Release Planning and Tracking in the Large 150Chapter 13: Concurrent Testing 155
Introduction to Agile Testing 155
Agile Testing Principles 156
Unit Testing 158
Acceptance Testing 160
Component Testing 162
System and Performance Testing 162
Summary: Agile Testing Strategy in a Nutshell 164Chapter 14: Continuous Integration 169
What Is Continuous Integration? 169
Continuous Integration 171
The Three Steps to Continuous Integration 172
What Is Continuous Integration Success? 175Chapter 15: Regular Reflection and Adaptation 179
Iteration Retrospective 180
Release Retrospective 184Part III: Creating the Agile Enterprise189Chapter 16: Intentional Architecture 195
What Is Software Architecture? 195
Agile and Architecture 197
On Refactoring and Systems of Scale 201
What Are You Building? 202
An Agile Architectural Approach for Enterprise Class Systems 203
Building Architectural Runway 204Chapter 17: Lean Requirements at Scale: Vision, Roadmap, and Just-in-Time Elaboration 213
Overview: The Requirements Pyramid 213
What’s Different About Requirements in Agile? 217
A Scalable, Agile Requirements Approach: Vision, Roadmap, and Just-in-time Elaboration 222
Summary 235Chapter 18: Systems of Systems and the Agile Release Train 237
An Agile Component Release Schedule 238
The Agile Release Train 242
Release Train Retrospective 247Chapter 19: Managing Highly Distributed Development 249
At Scale, All Development Is Distributed Development 249
Case Study 1. Ping Identity: The Distributed Define/Build/Test Component Team 251
Case Study 2. BMC Software, Incorporated: An Agile Transformation in a Highly Distributed, Large-Scale Enterprise 255
Emphasizing Communications 261
Tooling Infrastructure for Enterprise Agility 265
Summary 269Chapter 20: Impact on Customers and Operations 271
The Benefits of Agile Methods to Sales and Marketing 272
Impact on Product Marketing/Product Management 273
Smaller and More Frequent Releases 275
Optimizing the Agile Release Process 276
Real Challenges and Misconceptions Regarding Agility from Real Sales and Marketing Executives 284Chapter 21: Changing the Organization 289
Overview 289
Why Does Agile Require Organizational Change? 290
Preparing for Scrum and Agility 295
Eliminating Impediments to Software Productivity 298
An Agile Model for Executive Management 299
Rolling Out Scrum/Agile in a Large Organization 304
Summary 309Chapter 22: Measuring Business Performance 311
Agility Measures: The Key Difference 311
Measuring Team Performance 312
On Metrics, “Process Police,” and Team Self-Assessment 318
Scaling to Organizational Performance: A Balanced Scorecard Approach 319
Agile Metrics at Scale: Implementing a Flexible, Automated, and Meaningful BSC for the Enterprise 322Conclusion: Agility Works at Scale 325Bibliography 327Index 331
Forewords & Introductions
Software Methodology Career Phase I: Experiences at RELA and Requisite, Inc.
I have been committed to improving software engineering and software development management practices throughout my career. While that goal has been a constant, I now recognize three distinct stages of my understanding of software development practice and maturity. In Phase I, I was the CEO of RELA, Inc., where we developed software for others on a contract basis. RELA developed a wide variety of software applications ranging from stomach-churning adventure park rides to life-sustaining medical devices. Because the software we wrote was always for others, we were constantly aware of the need to “build the right thing.” Our individual livelihoods, our company, and its stakeholders all depended on our ability to understand what problems our solutions needed to address and how to apply effective best practices in achieving that solution.
In order to do so, we depended on many of the rigorous practices based on the waterfall method in use at that time. Indeed, some of our customers, and other key regulatory agencies such as the FDA, mandated its use, so we followed it prescriptively even as we tried to improve it. While it is entertaining now for many of us to criticize and make fun of the methods we employed, the fact is that the waterfall method was a substantial improvement over the cut-and-try methods of the past, and more importantly, it delivered results. Much of my focus at that time was on the requirements process, because that is where the critical discovery happened, solution behavior was defined, and our contracts were based.
That experience led me to my next career asfounder and CEO of Requisite, Inc., makers of RequisitePro, a product solution for requirements management. At Requisite, we advanced and developed requirements practices and products, so in a sense we became experts in the front end of the lifecycle. We sold Requisite to Rational in 1997, and I embarked on Phase II of my software development process career.Software Methodology Career Phase II: Experiences at Rational Software
In this phase, I was a senior executive at Rational Software and was involved in the promulgation of the Unified Modeling Language (UML) and the Rational Unified Process (RUP). At Rational, I had the good fortune of working directly with thought leaders such as Grady Booch, Ivar Jacobson, James Rumbaugh, Walker Royce, and Philippe Kruchten. During this time, my coauthor, Don Widrig, and I also published the first edition of the Addison-Wesley text Managing Software Requirements (2000).
Then our thinking was based on object orientation, and that technique provided additional flexibility in our development methods and additional resiliency in the software we wrote. It also led to a software process that was fundamentally different from the waterfall method and one that is characterized as being iterative and incremental. In this method, each iteration was a working piece of code that could be objectively assessed and evaluated. This method was far more agile than my prior experiences: we were no longer forced to rely solely on intermediate work products—documents, design reviews, and the like—but could see and measure tangible progress.
Rational codified that process in a written process description, the Rational Unified Process, and marketed and applied that process with good success across the industry. In addition, we applied the process in the development and release of Rational Suites, which required the coordination of as many as 800 team members in four countries. We released Rational Suites twice per year, each with an integrated set of products and a common install. Rational was eventually purchased by IBM, and today RUP is marketed under the auspices of IBM’s Rational Software Division and is in use by hundreds of thousands of practitioners.Software Methodology Career Phase III: Experiences with Agile and Rally
Upon leaving Rational, I became an independent consultant and adviser to development-stage software businesses, where I coached business strategy and software development practices to a half dozen new ventures. I used the opportunity to leverage some of the more innovative, lightweight methods, including XP and Scrum, and witnessed firsthand the improvements in productivity and quality that these methods delivered to these smaller teams.
After a short time, I was so won over by these methods that I soon refused to engage with any business or any team that did not have a strong sense of agility. The risk to the business was too great otherwise! At the same time, I began to see the limitation of those methods. As the teams and applications grew, the team’s ability to refactor code became less practical, and we also noticed the need for more assured communication of requirements as well as for more “architectural runway.” During this time, I was also consulting methodologist to Rally Software and helped to develop its hosted solution for distributed agile development. At Rally, I was heavily influenced by our interaction with agile thought leaders such as Ryan Martens, Ken Schwaber, Jim Highsmith, Mike Cohn, Tom and Mary Poppendieck, and Jeff Sutherland. Experiences with Agile at Enterprise Scale
Concurrently, I was personally challenged by a number of larger organizations to bring the level of agility and responsiveness I had witnessed to their enterprise. It was with some trepidation that I accepted this assignment and spent the next few years applying the core principles of agility in larger organizations, including applying my experience to large-scale development at BMC Software, Inc., where we worked with hundreds of highly distributed developers to deliver new applications of substantial scope and scale.
In so doing, I was pleased to discover that many of the best practices as taught by the agile methods delivered immediate out-of-the-box value to the enterprise. I also discovered that, by themselves, these best practices did not fully address the challenge at enterprise scale. Therefore, we gradually evolved an extended set of practices that were necessary to achieve enhanced agile results at scale. Finding little published in the marketplace to counsel larger companies, I resolved to write this book. I do so in the hope that your enterprise can leverage what we’ve learned and apply it to deliver higher productivity and higher quality to your customers. In a world dominated by software, it is hard to imagine a higher leverage point for our industry and, indeed, our economy as a whole.How to Read This BookPart I: Overview of Software Agility
This book is divided into three parts. Part I provides a short history of the agile movement with a discussion of some of the primary agile methods that are in use today, including XP and Scrum, as well as a discussion of RUP, which is an iterative and incremental method that can be applied in an agile fashion. In addition, we take a brief look at a number of other methods that helped sponsor the agile movement, including Lean Software Development, Dynamic System Development Method (DSDM), and Feature-Driven Development (FDD). We look at these methods not to teach the methods themselves but to provide a basis for understanding Parts II and III. As we will discover, each method has brought substantially new thinking to software development practices, and each has contributed substantially to the state-of-the-art. In addition, we’ll start to see a set of agile best practices emerge, many of which have already been applied at significant scale, and we will use these as the basic building blocks of enterprise agility.Part II: Seven Agile Team Practices That Scale
Part II describes these building blocks, the seven agile team practices that scale, one per chapter. In a sense, these practices may be considered the essence of agile in that all the agile methods apply these practices either explicitly or implicitly. For those new to agile or for those large organizations contemplating implementation of these practices, Part II of the book should provide comfort, because by simply adopting any of the agile methods described—or better, mixing and matching as necessary for the company’s current context—many of these best practices will naturally emerge and provide an immediate benefit to applications of virtually any scope. While they are not trivial to address and master, they have been proven in a wide variety of project contexts, and they will benefit any team that adopts them.
Together, Parts I and II of the book provide an overview of software agility and describe seven best practices that can be applied at virtually any scale. Each of these practices can directly and immediately improve the productivity and quality outcomes for teams who choose to adopt them.Part III: Creating the Agile Enterprise
To achieve true enterprise agility, however, more work remains, and that is the topic of Part III. We describe an additional set of capabilities, guidelines, principles, practices, and insights that will allow the organization to apply agility at virtually any level of application or system scale. These practices have been derived from experiences in the field in applying agile in larger circumstances. They include “green field” projects of smaller teams of 40 to 50 developers distributed across multiple countries, including extensive outsourcing, as well as larger organizations of up to a thousand developers working toward a common purpose on systems that required intense coordination among those teams. Some of the principles in Part III may seem obvious at first. Others are more subtle and have been discovered by experience in applying agile at scale. Many of these principles came about as teams reflected on their prior release efforts and came to modify their behavior over time so as to continually improve results.
Taken together, our hope this is that this book will help larger organizations achieve productivity and quality gains of as much as 200 percent, as such has been achieved by the smaller teams that have applied them. In turn, these results will provide benefits of faster time to market, higher return on development investment, and increased customer satisfaction for the enterprise. And lest we forget, organizations that head down this path have an additional intangible benefit: the teams themselves love agile methods, and empowering them to experiment and advance their methods is a key to engaging them in a virtuous cycle of empowerment, continuous process improvement, improved project outcomes, personal and professional growth, and higher job satisfaction. In an industry that faces the challenge of encoding much of the world’s intellectual property, what could be more virtuous than that!