How can Agile kill a project

I'll continue to talk about software engineering from previous post (How can Waterfall kill a project). People can learn from the failure of many waterfall projects in the wrong way. Let's see how agile can go wrong in the same situation.

A company has come up with this great idea to build car, boat, and truck (assuming they never exist before) to get to the emerging market. Being an innovative company, they really want to push the technology boundary.

Learning from the failure of the waterfall process, they decided to adopt agile methodology and build cars first without worrying about boats and trucks. With this simplified requirement, the project went smoothly. They delivered cars to the market on time within budget. Everyone is happy.

When the market starts to heat up and competition pour their products into the same market, manufacturing only cars is no longer profitable. So the company decide to continue their adventure by building the boats. With the success in delivering cars, one would reasonably think building boats should be much easier until one developer points out that they require about the same amount of the budget for building cars. Product manager asks why and the developer provides this answer: "Developing the engine is the part that consumed majority of the resource. The engine built for the cars is embedded into the car frame and cannot be reused for the boats since the car frame is not suitable for building the boats. To develop the boats, we'd have to develop the engine again." The company did their calculation. If they invest in building the boats with about the same cost, by the time their boats hit the market competition may also deliver similar thing. Since manufacturing cars is no longer profitable, manufacturing boats with the same cost may suffer the same fate. So the decision is made not to build boats less to say trucks. Since manufacturing cars is no longer profitable, the manufacturing line is closed. Everyone is sad.

In retrospect, what could have been done differently? Why didn't they design an engine that works for cars, boats, and trucks? Wait a minute! Are we going back to the failed waterfall process?

I'd learn the lesson differently. Agile tends to give developers tunnel vision and leads to under-engineering of the product. Taking away the design step in the engineering process only makes this situation worse. Agile or not, you still should follow the same design/engineering principle: separation of concerns. Had the engine been designed in such a way that it is not concerned about how it is mounted (in this case embedded in the car frame), it could have been reused for the boats even though it may not be capable of driving a truck yet.

How can Waterfall kill a project?

Waterfall engineering process has been used in many industries with great success. We've used it to build airplanes, railways, buildings, and many other things. But in software industry "Agile" has become THE software engineering process everyone is following. What is wrong with waterfall?

I often use this simple story to explain my understanding of it.

 A company has come up with this great idea to build car, boat, and truck (assuming they never exist before) to get to the emerging market. Being an innovative company, they really want to push the technology boundary.

The project is following a waterfall model. They started by doing a good design of the things they want to build. During this process, they realize that they need a piece named "engine" that can be reusable for all three products. They start to design this perfect engine. Nobody has ever built something like this before. Invent an "engine" is hard and invent an "engine" that can fit into all three products is harder than that. A lot of resource is put into the effort. The resource required is so immense that the project runs out of budget before they can deliver a single car.

In retrospect, what could have been done differently? Why didn't they design an engine that only works for cars and deliver some cars before worrying about boats and trucks? One would reasonably believe designing a simplified "engine" just for cars should not have consumed so much resource. In terms of software engineering, the lesson learnt is waterfall tends to over-engineer a product or a part of it. The effort put into this over-engineering effort may cause delay for accessing the market.

Some people learn the lesson differently. They think the "design" piece in waterfall is to be blamed for the failure. Why do you spend so much resource on generating these useless design documents before writing code? Coding is design after all!

 I believe lessons learnt this way is part of the reason that "agile" is born. In the name of "reducing documentation overhead", the design step is removed from the engineering process so that no design documents are generated. But I don't agree with the way this lesson is learnt. Can Agile kill a project? That'll be my next post.