Twitter follow Button
(@NetlabLoligo)


 


  		 



Managing IT:
Agile Bridge Building
A gently satirical look at agile software & extreme programming (XP) development philosophies


Contents

...with proper design, the features come cheaply. This approach is arduous, but continues to succeed.
  -Dennis Ritchie
[top]
Overview

This paper presents a lightly satirical look at the latest fads in management of software design and development projects. It provides a high level overview of some of the problems with pattern based software philosophies and tries to familiarize you with much of the terminology currently being used to describe them. The latter is a fast moving target though.

My aim is to show you that your hard learned understanding about what it takes to do a job well, on time, and within budget, is just as applicable to software development projects as it is to other types of projects. You'll see that everything you've ever learned about bringing projects to completion has not suddenly become obsolete.

Let's start with what you already know: It doesn't matter if you're building[1] software, a bridge, a space shuttle, or even something as seemingly trivial as a world class meal. The simple principles at the heart of producing any "job well done" are constant. They need not be a mystery to anyone who has ever performed a difficult task from inception to completion, and performed it well.

So this article attempts to have a little fun while exploring what might happen when you apply my vocation's pop project management philosophies to another type of task; bridge building[1]. Perhaps as software developers, we can laugh at ourselves a little along the way. What better way to learn?

Presented for your approval...





[top]

The New Way to Build a Bridge

In this article the author describes a bridge building project to demonstrate the superiority of agile bridge building methodologies to old, outdated concepts that rely on Big Up Front Design. The initial goal for the project will be to produce a river crossing allowing traffic and commerce to cross over the bridge while (ideally) allowing river traffic to pass under the bridge.

It should be noted that the author's choice of the term "initial goal" is appropriate here, for it is a foundational principle of the agile bridge builder's manifesto that the client should always have the power, freedom, and flexibility to change his or her mind at any stage in a project's ongoing development, right up to completion. Put another way, everything is tentative, no decision should be written in stone.




[top]
Rapid Span Development

The bridge-building masters on the rapid bridge development team begin by floating a log out on the water, half on land and half into the water. This does not (yet) accomplish our project's goal, which is to build a river crossing that ships can pass under, but it puts a working span out on the water quickly.

Allowing bridge investors and executives to see working wood beginning to make the river crossing is an extremely important concept in extreme bridge building (XB). It shows the project has immediate success. This in turn helps to mitigate the hurtful effects that nervous or less confident executives may have regarding our ultimate success. Having placed a working span of bridge part way out toward our goal, we may now refer to our efforts with undeniable correctness, as a successful bridge design and development project[2].

This rapid approach combines design and construction into one easily comprehended step. It completely avoids all of the Big Up Front costs associated with the lengthy and COSTLY bridge design phases that were once used, back in the (old) days when people designed bridges before they built them.




[top]
The Self-Realizing Project

We have now embarked on Iterative Incremental Bridge Design (IIBD). A cornerstone of IIBD is how new design elements will suggest themselves the instant you begin building. For example, it is only by building this first section of the bridge that we are able to see that the log is not totally stable and could easily be washed away by changing water levels. So a team, perhaps a pair of bridge builders, may thus be intuitively assigned to rework, reinforce and better secure the first log to the embankment. These opportunities to "re-factor" elements of our design will present themselves often within the course of our bridge project and should be pursued relentlessly.

Similarly the simple act of starting the bridge's construction exposes other design opportunities that seem to magically suggest themselves. For example, one will realize, with no need to engage in any wasteful Up Front Design oriented thinking, that the single log does not span the entire width of the water. So, almost as if by second nature the next design decision becomes apparent: Have another pair of bridge builders travel to the end of the first log and lash a second log to the end.




[top]
BUFD Is a Bad Thing

This effortless ability on the part of the builders to produce viable design steps at the exact same time the building effort commences is partly due to the synergy from the mystical chi of the bridge project itself. But be careful! If the team's Yin and Yang are not in nearly perfect balance your bridge building pair will get wet. This of course is not the result of the project or methodologies used, but the fault of the individual team members. They simply aren't disciplined enough to infer the patterns that emerge from the bridge construction effort. Simple attitude adjustment can frequently address this. Sometimes mocha latte is enough. Or an agile bridge building guru can come in to assist in inferring the pattern and facilitating the acceptance of chi.

Developers who don't exhibit proper chi may be better off doing projects with the old-fashioned Big Up Front Design dinosaurs (or "BUFD's" ). But if you need a bridge we recommend you steer clear of BUFD developers. They are unfamiliar with the Zen and pattern of agile bridge development. They will invariably force clients to wait until a design is in place before allowing them to see their projects taking shape. The design phase can often take longer than the building phase. Worse, some of these BUFD bridge builders will charge for the cost of design.

If you've been paying attention, you can now see that design is something freely performed by the project itself when you apply the principles of extreme, guru, master, wizard, bridge building to the task at hand. Today, thanks to the agile methods of IIBD, those touchy-feely psychobabble concepts like trust and delayed gratification need no longer have any place in the serious contractual business of bridge building.




[top]
IIBD Is a Realistic Approach

Of course, many other problems will arise as the bridge is being built and designed at the same time. But minor issues will likely be surmounted over the course of the construction effort. It is more important that clients are able to see working spans over the river almost from the very first day of the project. Agile bridge developers do not force clients to wait around for costly and burdensome design stages. Nor are clients asked to sit in endless meetings as designers attempt to determine what is really needed.

In agile bridge development, after all, the circumstances of the ongoing construction automatically show clients what they really need. Meetings, if they are needed at all, will be stand-up meetings.




[top]
No Decision Should Be Final


Which brings us to our final undeniable point about the benefits of IIBD - total flexibility. The client is always free to change his or her mind at any time. Because the design is taking shape simultaneously with the actual construction of the bridge, there is no disconnect between the designer's vision and the customer's understanding of that vision. As the bridge takes shape, the customer sees the design trade-offs played out in real time, and can make his or her own decisions on each one.

In this project for example, it quickly becomes clear that river currents are pulling the far end of the bridge downstream. The client can see for himself --without being told-- that more time and resources will be needed to complete the span.

Likewise, the client in this case will be able to see that the original design goal of allowing boats to pass under the span was excessive. The new bridge will allow cargo to be transported by land vehicles, doing away with the need for boats altogether.

This saves much of the COSTLY bickering and bad feelings (not to mention lost business) that may be associated with having a designer explain this to the client Up Front during one of those long unnecessary design meetings. Also, no valuable developer time will be wasted designing this feature that --it turns out-- wasn't really required after all. In short, we were able to say to the client at the most opportune time: "You're Not Going to Need It."




[top]
The Old Way to Build a Bridge

Bridge builders who have not kept their skills current continue to build bridges in the old way.

Recommended Viewing

video jacket cover showing bridge Nova: Super Bridge
Note: Though this paper has taken a lighthearted tongue-in-cheek approach, I really do recommend this video and believe it will be very enjoyable for anyone who just loves it when a plan comes together.

This video by NOVA relates how the old way of building bridges costs its investors and practitioners a great deal of time and money.

Disadvantages documented in the video include:

  • An extensive planning phase requiring huge commitments of time and money from those commissioning the bridge.

    Hundreds, of sit-down meetings, tens of thousands of designer hours, millions of dollars, all, as we have shown, unneeded. They even had the Army Corps of Engineers build a scale model of the river section that the bridge would cross, complete with scale currents and flows reproduced from measurements taken of the actual river. The flows would have been immediately apparent if they had just started building!

  • Unrealistic demands on those constructing the bridge to know their craft and follow the plan precisely.

    For example, the two center towers had to be placed within 1/8 inch (.32 centimeters) of their planned position in the middle of the river. This was an incredibly expensive exercise, which was dangerous, and difficult.

    As all Extreme Bridge Builders know, it is unrealistic to expect bridge builders to learn their chosen vocation to this level of proficiency. For those who don't know, here's a pointer; requiring bridge builders to know this much about bridge building is often pointed to as a cause of many defects found in bridge development today. So hiring developers with such capabilities should be avoided.

    And of course, agile development might have shown there was not need for towers at all so all this Big Up Front design is not only completely unnecessary anyway, but actually quite wasteful.

  • Unable to meet schedules despite all the time wasted in planning and scheduling.

    This is perhaps the most compelling argument for Iterative Bridge Development methods. Had they employed agile bridge development methods, they would have at least had some working usable span over the river from the very first day. Not finishing by some arbitrarily chosen drop-dead deadline was very COSTLY (to the developer) as documented in the video.

    There's more, but to truly appreciate the full impact of using old, non-agile styled bridge-building techniques we recommend watching the video.





[top]
Dominic's Law:
Easier to design is harder to use more than half the time
   -D.J. Repici

Back to Reality

I hope you have enjoyed this little tale of two bridges, and that it might help you to recognize when you are being asked to excuse bad development practices simply because "software is different."

We in the trade have lots of excuses and reasons but none of them really ring true. We will try to tell clients that software is different because you can't touch it, and therefore there is no real cost to make changes in software constructs. We know nothing could be further from the truth.

. . . . .
Pliability Is Everywhere

You don't have to be an economics wizard to understand that ultimately almost all of the cost of constructing a bridge (or a meal, or a skyscraper) is the cost of the labor and organizational efforts that go into converting raw materials into finished products. The value of the touchable materials that come out of the ground or forest is minuscule compared with the costs of handling them all the way onto and into the bridge. Intangible costs begin with the costs involved in digging (or growing) the tangible things. Intangible costs continue to be added at every step along the way to the bridge.

Cost issue 1 - It is all labor.

. . . . .
A Tower in Any Other Form

There are other costs associated with not planning a project before starting. One problem is, if you don't know what the finished construction will look like before you start building, you must wait until it's completed before you can see that a particular component doesn't make the grade. Maybe that first plank should have been painted before it was put in the river.

"So what?" we will say, "In software development it is easy to replace a component, even after the entire project is complete. All that is spent is the cost of developer time to re-write the code." As explained above, labor is everything and developer time is often the single biggest expense item in software projects.

This added cost, due only to lack of planning, would be unacceptable by itself, but it is not the only cost added. This is because the one component you replace is related to every other component in the project, and changing it will affect all those related components.

Cost issue two -

In software, as in bridge building, there is a domino effect. Each successive component often depends on the capabilities of the components below it. Reworking one low-level component will likely require reworking many of the dependent components that have been built above.

. . . . .
Whisper Down the Lane

And the hemorrhaging caused by lack of an easily consulted plan goes on. Consider testing! I will be the first to tell you, testing is a good practice, and the tests themselves might be well-designed and carried out with great precision. A component may have tested fine for what testers thought was needed at the time of the test. But no matter how thorough the process, without the ability to reference a detailed and shared understanding of how the final project will use each component, the tests can not be relied on. They will be tests performed in total blindness.

But testing will give everyone a secure feeling, which will make it less likely that anyone will go back with any conviction to re-check these fully "unit tested" components. For one thing, retesting components that have already been thoroughly tested would correctly be flagged as a redundant expense by management.

The tendency not to return to fully tested components is further exacerbated among team members by the specter and costs associated with tipping over that "first domino" discussed above. That is, finding a deficiency in a low-level component that leads to the rework of hundreds of higher levels.

Third and final cost issue -

Many factors combine to greatly increase the odds that a project will be placed into service with components that have been fully tested and found compliant, but that are wholly inadequate. This risk can be effectively avoided simply by gaining a precise, easily consulted understanding of the completed project before component construction and testing begins.

. . . . .
It's About Time

Perhaps the most transparent excuse we programmers clutch to is that software is different from other design disciplines because humans haven't been doing it as long.

In the case of a satire about bridge building, for example, we will say that people have been building bridges for thousands of years. But people have only been building modern steel-cable suspension bridges since the mid 19th century. Discounting the fosil record, that would be right around the same time Ada Byron wrote the first modern machine instruction program. Whether we are talking about Rosetta Stones, or trees felled across a river, we are splitting hairs at best.

So to bring the point home without need for hair-splitting on either side, consider that we've only been designing winged flying-machines since around 1900, and that the above satire would work just as well if it were about "Extreme Jumbo-jet Building."

We who have been foisting so much change on others lately seem to be having a little trouble with the concept ourselves. Suspension bridges, jumbo-jets, integrated circuit design, and so much more, are all life's way of telling programmers: it's time to change now.

. . . . .
If You Fail to Plan, Plan to Fail

The documentary video recommended above describes an actual case where implementers had to change concrete to meet the plan. In one section, the concrete poured did not meet the requirements documented in those big up front specifications. Observe that without the big, detailed, up front specification, they could not have known what capabilities would ultimately be required of the concrete, and so would have had no way to know that it did not meet those capabilities.

In other words, without a big up front design this team could not have learned of the concrete's shortcomings until after the bridge was built; possibly not until it fell down. It is for this reason, and not because concrete is hard to change, that bridge builders put big up front effort into their bridge designs.

Is software really different? In the real world, the costs in resources and risk associated with software projects that fail after they have begun are remarkably similar to those associated with other types of projects. That is to say, they are stunningly high.



[top]

We Know How to Do This

The best chefs will always experiment and improvise to come up with new and better ways to build a world-class meal, but not while serving diners who have come for a world-class meal. The best building and bridge designers will continually experiment with new construction techniques and new materials combinations to improve their craft and designs, but not while building a project counted on to be safe and reliable. Even comedians (can you imagine anything more pliable than a comedy act?) will do improvisation and try out their new material, but only in the hard-core dives that are the purview of the most serious comedy enthusiast. And then, only when the customers understand that their purpose is to be guinea pigs (e.g. open mic or improv night).

Trial and error is good. It improves us as individual practitioners, and it improves the profession as a whole. But it should not be sold as a product to people who expect to use it in practical and sometimes critical areas of their lives and businesses. Engaging in such folly may tarnish the credibility of our profession.



[top]
Luck is the residue of design
    -Branch Rickey (former owner Brooklyn Dodgers)

Left Foot First

In our early years, we in the field of software development learned a great deal about software design and structure. It is time, and we are ready now, to make and keep schedule commitments, well-planned "big up front designed" specifications, and project budgets. How can we be sure it's time now? By observing how silly our excuses for not doing these things have become.

. . . . .
Three Simple Questions

The following three questions should be asked by anyone requesting development work of any kind and should be answered to the requester's satisfaction by anyone performing that work.

  1. By what date (or date range) will the project/phase be completed?

  2. What exactly (itemized) will be provided by the date(s) specified in question 1?

  3. How will you guarantee that the items specified in question 2 are delivered by the date range specified in question 1?
    (In other words, what consequences will you accept if the specified items are not delivered by the specified dates?)

Additional items requested as development progresses may be specified and considered by re-asking and re-answering these three questions for the entire phase (with the new features included). Alternatively a new design specification may be opened specifically to hold new features to be implemented after release of the current phase.

Such a discipline may be applied to delivered software as well as exploratory services (eg. "We will provide answers to this specified list of questions", . . . or . . . "... provide requirments aqcuisition and analysis sufficient to produce an estimatable project brief for this described project", etc.).





[top]
Permissions

This article is © Copyright, Dominic John Repici 2003 - 2019 ALL RIGHTS RESERVED. It is used here with permission.

Links to this article are always welcome.

However, you may not copy, modify, or distribute this work or any part of it without first obtaining express written permission from Dominic John Repici Production and distribution of derivative products, such as displaying this content along with directly related content in a common browser view are expressly forbidden!

Those wishing to obtain permission to distribute copies of this article or derivatives in any form should contact D. John Repici at Creativyst.

Permissions printed over any code, DTD, or schema files are supported as our permission statement for those constructs.




[top]
Acknowledgements

As a software guy, I'll be the first to admit my human language writing skills need all the help they can get. I'm thankful for people who have responded to my calls for help in that department. This article owes a lot to suggestions and support from them. Joo KNOW who joo ah! I'm in your debt.

Extra special thanks to:

  • John O'Meara
    Who generously provided many edits and suggestions that have made this paper much less "wordy" and much more approachable. Your suggestions went beyond editing, providing a number of tips that have helped me to be a better writer. Thank you John.

  • Donald W. MacVittie
    Thanks for your support and especially for being the first to agree with the premise. :-) Thanks also for the helpful suggestions that improved the clarity of this paper.

  • Scott Miller
    Thanks for helping me find more flaws and guffaws.

  • Robin Laing
    Thanks for your suggestions and help with clarity.




[top]
Further Reading

Links to selected resources will be added here.

  • [PDF] - Extreme Programming Considered Harmful for Reliable Software Development 2.0
    This is a well balanced and well written paper that backs up its assertions with plenty of hard references. It debunks some of the claims of the XP movement. For me the most shocking revelation was the truth about the Chrysler Comprehensive Compensation (C3) project, which has long been related as an XP success story (see study below). This paper also explains that the equally exalted Ford VCAPS project is, in fact, another failed XP project. Google on "chrysler XP project" to get some idea of the level of hype and spin that project continues to get.

  • [PDF] - The Collaborative Software Process
    I present this as balance. It is a study designed to show that pair programming is more efficient than the traditional solo/colaborative techniques. The author has asserted that the experiment was "planned meticulously with [Clif Drew (utah.edu),] a highly-respected research designer" to be esentially free of any bias. Here's a sample passage from page 10 of the dissertation:

    The largest example of [XP's] accomplishment is the sizable Chrysler Comprehensive Compensation system launched in May 1997. The payroll system pays some 10,000 monthly-paid employees and has 2,000 classes and 30,000 methods. Additionally, programmers at Ford Motor Company, spent four unsuccessful years trying to build the Vehicle Cost and Profit System (VCAPS) using a traditional waterfall methodology. Then, the engineers duplicated that system, this time successfully, in less than a year using Extreme Programming

    In fairness it must be noted that these are the same failed Chrysler Comprehensive Compensation (C3), and VCAPS projects that have been documented in the paper listed above. The C3 project in particular was canceled by Chrysler four months before this paper's publication date. It had never reached its promise of providing compensation services for over 80,000 employees.


  • Problems with agile practices
    From Efficient Software Development Methodology.


  • Another "Successful" XP Project: CSTARS
    In addition to the two failed projects related above that have been hyped by the XP movement as successes, there are others, such as this one named CSTARS. The contractor (Avanade) and the greater XP community will surely want to distance themselves from this one. Consider some bullet points taken from the actual press-release that announced this failure. (source: RockyMountainNews.com). According to the article the contractor:

    • Wrote computer code before they were given detailed plans for what that code should do.

    • Disagreed over who should make key decisions, including which side was responsible for compiling detailed plans.

    • Ignored the state and county staffs that would use the new system and fired the subcontractor in charge of seeking their advice.

    After six years and eight million dollars were spent, a "Department of Revenue director halted the system's most recent trial run [...] and brought in another consultant to see if it's salvageable." Not surprisingly (see Footnote #2 below) the Avanade representative claimed the project was completed successfully, and blamed any misunderstanding entirely on the client. Also worthy of note: Avanade is a Microsoft Partner and provides only Microsoft platform solutions.

    As stated the Agile/XP movement has become predictable in these matters. If the past is any guide, they will first try to spin it as a success (they have, see the article), and then distance their methods (they call them "methodologies") from the failed XP project. Can we take steps to head - them - off - at - the - pass?


  • Yet Another "Successful" XP Project: 737 Max 8
    Sadly, some time after the first publication of this article, some highly degreed "computer scientists" (whatever that is) actually did try to apply these extreme software development techniques to jumbo-jet software production. It is a testiment to how good these marketing types are at their real craft, that even after one plane went down, killing many, they were able to convince people to keep their software in the air. Over a hundred more died in this second plane crash.


  • The Date-Picker in Microsoft Edge
    Try to use the date-entry widget in Edge browser. I'm sure the computer scientist, or coder, or software consultant, or any other euphemism for not-a-real-programmer that came up with it thought s/he made a better calendar than the one we've perfected and grown accustomed to over the centuries. . . Who really needs to see the weekdays associated with the dates we're picking? Apparently not us! What do the XP crowd like to say? “You're not going to need it.” Count this as validation for Dominic's Law (“easier to design is harder to use more than half the time”).


  • Is the term "Waterfall Approach" an XP strawman?
    The saga continues... This is yet another report from the front lines of the never-ending search for the origins of this term. XP followers often claim that this is how software was developed before their methods arrived on the scene. My own reading suggests that the term is mistakenly attributed to Winston W. Royce's example of how software development is NOT done. It came from later descriptions (by Bell and Thayer) of his illustration that, in fact, showed what would be a disastrous method of software development ... if ever actually used. In this paper, he spends the remaining pages describing how the basic framework of figure 2 is modified in common practice to produce a workable (iterative!) development process. To quote Royce:

    "the implementation described above [in figure 2] is risky and invites failure."



  • Software Reality
    This website is for software designers who are looking for common sense solutions to their every-day problems. It is hosted by Matt Stephens, a coauthor of the book: Extreme Programming Refactored. As such it has become somewhat of a refugee camp for people who have been through the XP grinding mill and come out looking for a better way. People who enjoy shows like Monty Python, and books like HGTG will appreciate the very English brand of sharp-edged wit practiced here. Be warned though, the above article will seem almost pro-XP compared to some of the views expressed at this site.


  • XP techniques applied to wheelchair-ramp design
    They say a picture is worth a thousand words...

    Wheelchair ramp with two steps just before the bottom



  • Route 35: Millions spent on unexpected changes.
    Extreme road building. . .


Also, from Creativyst.com:





[top]
Notes
  1. Note regarding semantics           [return]
    The term "bridge building" is used in this article to mean everything that goes into a bridge-building project from:
    • the day the riverfront communities decide to commission a study on the need and feasibility of the bridge; equated to analysis and initial requirements acquisition in a software project or increment, through
    • the design phase; equated to design of the software project or increment,
    • the construction phase; equated to coding the components of a software project/increment,
    • and on to the day when the bridge is first opened to the public; equated to taking the project or increment live.

    Likewise the term "building software" in this context is not related to the basic compile/link cycle that is referred to as a "software build." In this article the term refers to the entire process of producing (building) a software solution, from the initial determination of whether it should be done (ROI studies, etc.) to the roll-out(s) (phase/increment go-lives).

  2. Note: How do you define "success"?           [return]
    At least twothree (see above) high-profile failed XP development projects are currently marketed by the XP movement as shining examples of XP's success. These are the Chrysler Comprehensive Compensation (C3) project and the Ford Vehicle Cost and Profit System (VCAPS). Because XP promoters and practitioners have this very forgiving idea of what constitutes a successful project, it can be difficult to judge how many other "successful" XP projects may have, in reality, been failures.

    To get some idea of the XP movement's ability to market their failures as shining successes, google on either of these two projects. To catch a glimpse of "that man behind the curtain" (i.e. the disappointing reality behind the hyped illusion, as per the Wizard of Oz), see the first reference in the section here titled "Further Reading" (above).
















 
© Copyright 2003 - 2019 Creativyst, Inc.
ALL RIGHTS RESERVED

Written by: Dominic John Repici
(with lots of help from good people)