Table of Contents
| Ch. 1 | Introduction | 3 |
| Ch. 2 | Dealing with complexity | 35 |
| Ch. 3 | Dealing with change | 87 |
| Ch. 4 | Paradigm shift | 109 |
| Ch. 5 | Software factories | 155 |
| Ch. 6 | Models and patterns | 193 |
| Ch. 7 | Programming with models | 231 |
| Ch. 8 | Language anatomy | 279 |
| Ch. 9 | Families of languages | 321 |
| Ch. 10 | Systematic reuse | 337 |
| Ch. 11 | Software product lines | 359 |
| Ch. 12 | Platform-based abstractions | 393 |
| Ch. 13 | Components and services | 413 |
| Ch. 14 | Mappings and transformations | 455 |
| Ch. 15 | Generating implementations | 483 |
| Ch. 16 | A software factory example | 511 |
| Ch. 17 | Frequently asked questions | 563 |
| App. A | Abstraction and refinement | 597 |
| App. B | The Unified Modeling Language | 611 |
Read an Excerpt
Software Factories
Assembling Applications with Patterns, Models, Frameworks, and Tools
By Jack Greenfield Keith Short Steve Cook Stuart Kent John Crupi John Wiley & Sons
ISBN: 0-471-20284-3
Chapter One
Introduction
If this industry doesn't deliver, it will lose the hearts and minds of business leaders, and they will look elsewhere to make their gains. Michael Fleisher
According to the Standish Group, businesses in the United States spend about $250 billion annually on software development, with the cost of the average project ranging from$430,000 to $2,300,000, depending on company size. Only 16% of these projects are completed on schedule and within budget. Another 31% are canceled, primarily because of quality problems, creating losses of about $81 billion annually. Another 53% cost more than planned, exceeding their budgets by an average of 189%, creating losses of about $59 billion annually. Projects that reach completion deliver an average of only 42% of the originally planned features.
In any other industry, results like these would generate a quick response aimed at restoring the bottom line. For at least the last 10 years, however, stakeholders have looked the other way, letting corporate earnings generated by the strong economy absorb the costs of these failures. In these less certain economic times, they are now calling the industry to account for underperforming technology.
Are these problems of interest only whenthe economy is weak? Will they recede into the background again, as the market rebounds, or are there more pervasive and fundamental forces at work? For quite some time, they were cast against a backdrop of rapid innovation. No one seemed to mind if we struck out twice for every three times at bat, as long as we hit it out of the park the third time. Recently, however, progress seems to have slowed significantly. Is the engine of innovation running dry, or is something holding it back?
Tools Lag Platforms
We think the problem is that software development methods and tools have not kept pace with the rapid advances in platform technology of the last few years. Using the latest wave of Web service platform technology, for example, we can now integrate heterogeneous systems across multiple businesses located anywhere on the planet, but we still hand-stitch every application deployed on this technology, as if it were the first of its kind. We automate large abstract concepts like insurance claims and security trades using small concrete concepts like loops, strings, and integers. If the hardware industry took the same approach, they would build the massively complex processors that power these applications by hand soldering transistors. Instead, they assemble Application Specific Integrated Circuits (ASICS) using software and generate the implementations. Why can't we do something similar?
The problem is that stakeholder expectations tend to grow as platform technology advances. We have met increasing expectations for some time by finding ways to hone the skills of developers. Or, to put it differently, we have met the expectations of an increasingly computerized society in much the same way that artisans in the first few years of the industrial revolution met the expectations of an increasingly industrialized society by honing the skills of craftsmen. Up to a point, craftsmanship works well enough. Beyond that point, however, the means of production are quickly overwhelmed, since the capacity of an industry based on craftsmanship is limited by its methods and tools, and by the size of the skilled labor pool. We think the software industry will soon face massive demand on a scale beyond anything we have seen before, driven by a blistering pace of platform technology innovation, not only in the business application market, but also in the markets for mobile devices, embedded systems, digital media, smart appliances, and biomedical informatics, among others.
Currently, the most popular way to hone the skills of developers is to harvest and disseminate the practices of the most productive developers, as exemplified by the agile development movement [Coc01]. The value of this approach has been amply demonstrated. We question the assumption, however, that we can solve current problems, much less prepare to meet the demands of the next decade, by honing developer skills. Apprenticeship is a proven model for industries based on craftsmanship, but it does not scale up well. Observers have noted that the most productive developers produce as much as a thousand times the output of the least productive ones, while the least productive developers generally outnumber the most productive ones by a similar ratio [Boe81]. This means that there will never be more than a few extreme programmers, no matter how much we hone the skills of developers. Moreover, the practices of the most productive developers cannot be applied on an industrial scale because they are optimized for people with uncommon talents. We need a better way to leverage the talents of the best developers, one that can scale up beyond what that they can collectively deliver today.
Other industries solved problems like the ones we are now facing by learning how to rapidly customize and assemble standard components to produce similar but distinct products, by standardizing, integrating, and automating their production processes, by developing extensible tools and configuring them to automate repetitive tasks, and by managing customer and supplier relationships to reduce cost and risk. From there, they developed product lines to automate the production of product variants, and formed supply chains to distribute cost and risk across networks of specialized and interdependent suppliers, enabling the production of a wide variety of products to satisfy a broad range of customer demands. This is what we call industrialization. It is also the vision espoused by the pioneers of object orientation.
We are not suggesting, simplistically, that software development can be reduced to a highly mechanical process tended by unskilled workers. On the contrary, the key to meeting demand on an industrial scale is to stop wasting the talents of skilled developers on rote and menial tasks, so that they can spend more time thinking and less time doing housekeeping chores. Also, we must make better use of these few but valuable resources than expending them on the construction of end products that will require maintenance or even replacement when the next major platform release appears, or when changing market conditions make business requirements change, which ever comes first. We need a way to leverage their skills and talents by asking them to encapsulate their knowledge as reusable assets that others can apply. Does this sound far fetched? Patterns have already demonstrated limited knowledge reuse in software design [GHJV94]. The next steps are to move from the manual application of patterns expressed as documentation to the codification of patterns in frameworks, then to the assisted application of patterns by developers using tools, then to the encapsulation of patterns by languages, and then finally to the fully automatic application of patterns by language compilers. By encapsulating knowledge in languages, patterns, frameworks, and tools, and by reusing these assets systematically, we can automate more of the software life cycle.
Software components will not be as easy to snap together as ASICs soon, since ASICs are the highly evolved products of two decades of innovation and standardization in component packaging and interface technology. Software components are moving in the same direction, however. The Web service technology that we mentioned earlier represents significant progress in packaging and interface technology for software components. There is much more to be done, of course, as we will demonstrate in Chapter 13. Building reusable software components is challenging, and assembling them generally requires adaptation or changes in the components themselves.
We think there is room for significant innovation in these areas, however, such as using models and patterns to automate many aspects of component assembly, and using aspect-oriented methods [KLM97] to weave contextual features into functional ones, instead of rewriting the functional features repeatedly by hand for every new context that we encounter. We think the software industry can climb the innovation curve faster than the hardware industry did by using specialized languages, patterns, frameworks, and tools in software product lines. Of course, the hardware industry has the added burden of engineering the physical materials used for component implementation, while we have the luxury of building with bits. At the same time, the ephemeral nature of bits creates challenges like those faced by the music industry, such as the protection of digital property rights.
The Software Development Landscape
In this chapter, we want to set the scene for the discussion of critical innovations in software development methods and practices in later chapters. To justify our call for a new approach to software development, we must demonstrate our understanding of the challenges that software developers face today. Their world, indeed our world, consists of rapidly evolving platform technologies, increasing customer expectations, a seemingly endless need to acquire new skills quickly, and intense pressure to extract value from older applications while realizing strong returns on investments in new technology. At the same time, few developers are fortunate enough to build green field applications. Most developers must also deal with challenges presented by existing application portfolios and platform technologies. To keep the story simple and easy to follow, we tackle these two sets of challenges separately.
First, we explore the changes in platform technology and stakeholder expectations that created new kinds of applications and produced the portfolios that businesses hold today. To support this study, we introduce a fictitious company, and show how it might have grown over the last 30 years, describing the evolution of its application portfolio and platform technologies. We will follow this fictitious company throughout the book as it adopts the Software Factory approach.
Second, we analyze new platform technologies that are now emerging, identifying the new challenges they create and the existing ones they exacerbate. Here, again, we use our fictitious company. We walk through a skeletal set of requirements for an application that must be developed to support its relationships with customers and suppliers. We then describe the architecture of this application in just enough detail to expose some of the challenges that the company encounters using the emerging platform technologies. At the end of this book, we will rebuild this application using the Software Factory approach to show how it compares with current methods and practices.
Platform Technology Evolution
Our fictitious company manufactures tools and equipment used in the construction industry. Its name is Construction Tools, Inc., or CTI. We will also need a fictitious software company to illustrate some of the ideas described in the book. It will be hired by CTI from time to time to help design and implement business applications. Its name is Greenfield & Short Software, Inc., or GSS. Neither the companies nor the applications intentionally resemble any actual companies or applications.
We will use the timeline shown in Figure 1.1 to help us keep our bearings as we tell this story.
As we shall see, earlier generations of platform technology supported applications that served users inside the business and the transaction volumes they generated, while the current generation, by contrast, supports distributed applications that serve large numbers of users outside the business, such as customers and partners, and the transaction volumes they generate. Each wave of platform technology pushed the scope of automation outward, from inward facing processes at the heart of the business, such as accounting and materials management, to outward facing ones on its periphery, such as supplier management and warranty maintenance.
The Batch Era
In 1968, CTI employed about 500 people, and had annual revenues of $250m. It was a typical medium-sized manufacturing business. Structured along conventional lines for the period, the organization chart for CTI is shown in Figure 1.2. The company employed a full-time data processing staff of 30 people, including 10 in operations who managed equipment and job scheduling for their small IBM(r) mainframe running OS/360. Notice that Data Processing reported to Accounting. This was common in the Batch Era. Expenses for computer equipment, development, and operations did not fall into well-defined categories, and were often buried as accounting expenses.
Batch applications of the 1960s and early 1970s were used by only a few people within the business. Clerks captured data on paper forms that were sent to data entry staff who keyed the data onto machine-readable media, such as punched cards or tape. Most batch applications automated existing manual business procedures. The goal was to gain productivity by exploiting the calculating power of the computer to reduce processing time.
The Materials Management and Shop Floor Operations applications at CTI were typical Batch Era applications. As machine operators milled parts for Hand Drills and Jack Hammers, they reported time and material usage to a clerk who compiled the information and sent daily summaries to the data entry staff. The data was fed into a daily run of each application. At the start of each day, the Shop Foreman received a report from which he could determine the status of each order and allocate work to machine operators. The Stores Manager also received a daily report identifying the materials to be trucked from the warehouse to the various shop floors around the factory.
Because they automated existing manual business procedures, most applications were associated with a single department. The resulting application portfolios looked like Figure 1.3. They contained multiple islands of data, each used by a small number of applications, or sometimes by just a single application. Data was often duplicated across applications and stored in a different file structure by each application, making it difficult to retrieve and process. In the early part of the era, data was stored mostly on tape, making it difficult to keep the islands consistent. For example, in addition to entering daily data for Materials Management, the clerks had to record operator job times on a separate form for the weekly run of the Labor Costing application.
Continues...
Excerpted from Software Factories by Jack Greenfield Keith Short Steve Cook Stuart Kent John Crupi Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
loading...