Agile and Design Evoluation

Agile is right in at least one way in that no matter how hard you try, your requirements do change over time. There are good reasons for requirements to change. Market condition changes, technology advances, and product scope changes are all good reasons for it.

Let's admit that requirements are changing and you are always shooting at a moving target. To handle requirements effectively, you need a good strategy to evolve your design to deal with the changes.

In agile practice, developers are often given a limited view of the product to start their implementation. The best design they can come up with will unavoidably only cover the variations within that scope. You may blame the initial designers for lack of abstraction in their design to handle future variations. But in reality there is nothing wrong with it since they are limited by the knowledge about future requirements. It is hard, if ever possible, to determine how abstract their design should be to cover possible future variations without knowing what will come up. A more practical solution is to evolve your design from concrete to more abstract.

One way of handling requirement changes is to constantly revisit your conceptual models in your design in every iteration of your agile process. Before you add more if-else statements to your functions, more enum values to your switch statements, or boolean flags to your data structures, you should really think about your conceptual models and your state machines. Are they still valid given the new requirements?

If-else statements are really behaviour changes. Theoretically, behaviour changes are supposed to be handled by polymorphism in good OO design. The need for more if-else statements is a good indication that you need more derivations to handle behaviour differences. More of your base classes should become abstract (pure virtual in C++). If-else statements may bandage your obsolete design but will lead to more grief for the future.

More enum values or boolean flags are really amendments to your existing state machines, exponentially unfortunately. As state space of your state machines expands so dramatically, it is unlikely your last state diagram will hold. Revisit these state diagrams to see how it should really be under the new requirements.

Given a set of requirements, come up with a good OO design is already a challenging task in most cases. Shooting a moving target only makes it harder. Without a good strategy to deal with requirement changes, it is unlikely your software project will end up with great success, at least from software design's perspective.

Agile: What it should be and what it should NOT be

Agile as a software engineering pratice is gaining popularity. However, it doesn't always meet people's expectation. Like everything else, it comes with Pros and Cons.

When it all comes down to it, what Agile is trying to against is so-called "overhead" in "traditinoal waterfall" software engineering approach. We may want to take a closer look at the "overhead". In traditional waterfall approach, we may have documentations for requirements and analysis, design at different levels, and code. What is considered "overhead" is the time spent to create and maintain these documents. This is clearly marked by the advocates for "face-to-face" communication in Agile. However, there exists certain documentations that are required throughout your project that worths the effort to maintain. An obvious example is architectural design and component design guidelines. Theoretically, the information in these documents can be communicated by "face-to-face" communication but in reality that is not the efficient way to communicate them in entirety. How can you talk about your architectural design precisely the same for many times? Development teams are often very dynamic. If you want your team to have a consistent understading of the architectural design, documentation is more than likely a more efficient way of communcation than face-to-face. Without architectural design documentation and component design guidelines, the pressure exercised by Agile approach often lead to broken architectural integrity and sometimes broken encapsulations at lower level. When the skill level of the developers differ, the problem can eaily spread through the enire code base. The consequence is not obvious in current and near future itertions. However, as they accumulate, your code base becomes harder and harder to maintain. The cost to add new features will increase. For Agile to be efficient, it should be guided by architectural design and component design guidelines.

The good side of Agile is to advocate "Test-Driven-Development". This is what it should be. If a project is started with a good architecture design and good test coverage, it is unlikely for broken architectural integrity to happen. Component design guidelines may be reflected in the design and implementation of test suites.

Like anything else, Agile may be abused as well. It is just like a knife. You may cut your fruit or your finger depending on how you exercise it.

Agile should NOT be used as an excuse for lack of long term vision of product management. Agile advocates "embracing changes". However, it does not warrant product management the freedom to change their mind at free will often as a result of the lack of vision. In software development, changes are costly regardless what software engineering approach you take. You waste the time to put the code in and the time to remove unused code less to say the test code that comes with it.

Agile should NOT be used against the pursuit of a relatively stable architecture. Among all the costly changes, architectural change is probably the most costly. Without architectural design and component design guidelines, your code base is unlikely to be designed for changes since developers are rushing to finish current sprint without any consideration for things outside current sprint. Without architectural design and component design guidelines, you do not have the right tool to really embrace changes. That is exactly against what Agile has advocated in the first place.

For Agile to be sucessful, you really need to be agile. Learn from the lessons in your pratice and make changes as mistakes or negative effects are observed. That is the real spirit of the word "agile". Religious belief is just the opposite of it.

More Design Patterns?

Since the GoF, there are quite a few more publishments on Design Patterns. However, none of them can be as influncial as the GoF. Don't we need more design patterns?

In my opinion, the GoF revealed some of the fundamental principles/criteria for a good OO design. The key is how to distribute information/data and responsibility/code among different classes/modules in such a way that
1. foreseeable changes are made easy and
2. conceptual behaviour and actual implementation are decoupled.

Design patterns are examples where these principles are applied properly. Once you understand these principles and are able to apply them to real world design issues, more examples/patterns will certainly help you to enhance the understanding of these principles but the impact is becoming marginal. That is probably why none of the later publishments can be as influncial.

Agile without Architecture?

Being Agile is good but without a collective architectural view of the product, it may easily become cowboy development where each developer just does what he/she thinks is right. When that happens, architectural integrity can be easily lost. In this sense, the function of an architect is essential to the success of Agile development. This function may be performed by an individual or by a collective effort.