Recently, I came across the article, "Is Design Dead?", from Martin Fowler's blog. Martin addresses the common misconception that design is discouraged in Extreme Programming. He describes the overall purpose of design in software and how the design emerges as part of the development phase. As more is learned about the problem domain, better decisions can be made and the design evolves with the implementation. Refactoring code is an activity that helps facilitate success with this approach.
At a different time, I was learning about the structure of the human eye and some of the potential paths in evolution that could have led to the structure of our eyes today. I learned some surprising things with regards to how our eyes are structured. The evolved structure is counter-intuitive to what I would have imagined, especially when compared to the structure of a digital camera. I took a step back and starting thinking about the evolutionary path of the eye and compared it to typical events I have experienced during a software development cycle. I wanted to share the conclusions that I reached when I thought about these two topics.
Software Design Provides Flexibility
Design is an important aspect for any engineering discipline. Good designs allow for the development of a project to be divided into independent tasks. One of the most important benefits that can be realized from software design is the ability to remain flexible to change throughout the project. Software is so valuable for that simple reason, the flexibility and relatively ease of change. Now I say relatively because if the project is not developed and managed properly, the structure of a program can become rigid and more resistant to change.
The tendency to resist change as a project progresses is what drives the concept of the Software Change Curve. This curve says that each progressive phase of development becomes exponentially more expensive to change the software from its original design. A concept that may only cost one dollar to analyze may be orders of magnitude more expensive to introduce after the product as been released.
The Waterfall process spends a large amount of time up-front attempting to analyze and plan to the current known set of requirements. Time can also be spent to try to identify the most volatile areas of the requirements to create flexibility at the points that may change later in the project. However, this is still a difficult task, which requires the ability to predict the requirements that are likely to change. No amount of planning can foresee all potential issues that will occur.
The XP approach of evolutionary design asserts that it is possible to flatten the software change curve by making design corrections during the development process. Some of the practices in XP are intended to facilitate the corresponding changes in software to match the updates to the design; I am referring to testing, continuous integration (CI), and refactoring. Testing and CI help synchronize a teams efforts when integrating changes with each other. Software refactoring is the practice that provides the agility to efficiently alter the software by simplifying the volatile regions of code.
Structure of the Eye
So how does the eye relate to all of this? Let me first describe the structure of the eye, and I will draw the connection for the structure of the eye and its similarity to software development. In many ways, the eye is structured like any type of camera that has a lens. At the front, there is the lens, and the back of the eye is the retina. The retina is the light-sensitive tissue that detects light and sends signals to the brain for processing. The component in a camera that corresponds to the retina would be the CCD or CMOS image sensor.
The CCD sensor faces the lens and captures incoming photons and generates an electrical charge proportional to the amount of light that it receives. The retina functions in a very similar manner with two exceptions:
- Chemical and electrical nerves are triggered to send signals to the brain.
- The photoreceptors in the retina are at the back of the retina.
Let's focus on item two. The optical nerves that transmit signals to the brain come out of the retina facing the lens. Therefore, any light captured must travel through the layers of nerve fibers and other cells before they are detected by the photoreceptors. This also means there must be a location in the retina where the optical nerve passes back through the retina to the back of the eye and on to the brain, and there is; this point is called the Macula. It creates a blind spot in that area of vision for our eyes, that our brain fills in the missing piece, but that's another story.
Here is an illustration from Gray's Anatomy, which shows the forward facing part of the retina at the top, and the photoreceptors at the bottom:
The Code-and-Fix Process
The "Code-and-Fix" development process is what many software development projects devolve into at some-point in their lifetime; even if it's only a temporary situation. Basically, it evaluates what functionality is needed next, code the solution, and fix errors that are detected. Once a feature is completed, the next feature is implemented in a similar fashion. After a non-descript period of time passes, and numerous cycles of this process, the code base appears to have taken on a life of its own. The current project no longer matches any initial design work that may have been done before the project started. The source code has essentially evolved to meet the current needs.
Many people mistaken the "Code-and-Fix" process, with the practice of Evolutionary Design. That's because they are very similar to one another with the exception of one thing, Evolutionary Design prescribes that you must refactor regularly to simplify the current solution. By taking this step, your design decisions can be simplified by removing complexity from your system, and reintroducing flexibility. Remember, flexibility to change is what will help flatten the software-development curve. Refactoring is key to keeping your code nimble and adaptable to solve the next set of problems.
Back to Eyes
I now want to use the evolution of the eye as analogy of a "Code-and-Fix" development cycle without refactoring. I consider myself lucky to not require corrective lenses to see clearly, especially with so much time staring at a computer screen. Through my eyes, I think that evolution arrived at a pretty good implementation for an orbital ocular organ for me to sense light, track and focus on objects, and even adapt between high-contrast and low-light conditions.
Now consider how many millions of years vision took to evolve to this level. Also, the path that has led up to my eyes today, also has billions of other very similar versions evolving with each generation. The number of less-optimal mutations of the eye, which never came to be, must be orders of magnitude larger than the number of eyes currently on this planet. That is a lot of trial-and-error.
Refactoring
I think of "Code-and-Fix" as "Trial-and-error". This is a generally an expensive process to use with engineering. "Trial-and-error" becomes more expensive as the software development cycle progresses into the next stage. If you can remain cognizant of the issues that you have run into and apply this wisdom and experience towards altering the current implementation to be most beneficial to adapt to the next set of features, you have refactoring.
There is a little more to it than that, however, not much more. Refactoring source code is just like, well, it's just like factoring a fraction into a mixed number. You have a simpler form of the same thing. Refactoring is not intended to mean rework, re-implement, or create version 2.0. You are merely supposed to simplify the structure of your program while keeping the same functionality set. This is where unit tests are very valuable to help ensure you do not break things that were previously working. A good rule of thumb, if you have to change your unit tests when you refactor code, you are probably doing more than refactoring.
Regardless, you must continue to analyze and simplify the structure of your software if you want to keep the software development curve relatively flat. Otherwise, the code will become rigid, and resistant to the addition of new features free of defects. This is essentially the cause for the exponential growth of cost in the curve. Taking that little extra time to reorganize your code as you develop can payoff by allowing new features to be more easily developed. That makes more time for you do other things such as take on new challenges, or surf the Internet.
Summary
Evolution is a complex process that takes a lot of time, more time than we are given to complete a project. Evolution takes a lot of resources because it is a process that is based on Trial-and-Error. Code-and-Fix often is a Trial-and-Error process as well. There are more efficient ways to develop code. I would like to add that I do not think there is a single process that is suitable for every situation. Whether you design completely at the beginning, or let your design evolve throughout the development of your project, one of your goals is to keep your code base flexible. A code base that can easily adapt to new requirements and unforeseen circumstances, is a valuable code base indeed. The cost of development during progressive phases of development can remain relatively flat, as opposed to increasing exponentially. Because it will be easier to modify, easier to understand, and easier to maintain. Continual refactoring is a key to remaining nimble.
