Introduction
This is a story about how old-school software engineering principles failed me and how I learned to embrace change. I'm sharing it in the hopes that it can help others understand better understand one of the core principles of agile software development: responding to change over following a plan.
The confusion of agile
The principles of agile software development were first proposed thirteen years ago. Unfortunately today there is a lot of confusion around what it is and how it should be used. I attribute this in large part to the fact that it became an industry buzz-word where “agile” became synonymous with “good” and the ubiquitous two-day training on scrum processes watered down the original intent of the agile software development manifesto.
I think you have to truly understand the pain and failure of waterfall software development in order understand what motivated those seventeen software development gurus to get together and figure out why their industry was broken and how they could get it back on track.
A young programmer fresh out of school
Back in 1996 I was a young and idealistic software developer fresh out of school. I was eager to apply my new found understanding of software engineering principles learned at school to my job. At that time I worked in a neuroscience research lab where I created data acquisition and analysis software for a researcher and his colleagues. I wanted to do an excellent job and I was determined to make their tools into the best software they had ever imagined.
The big idea
I started my project by carefully gathering a list of requirements from my boss and getting him to sign-off on them. I then started off on a giant rewrite of the software he was previously using. It took me months to get to the point that I was proud enough of what I had done that I would let him use it. It had many of the features in the requirements document and many more that he hadn’t specified (but I just knew he would just fall in love with them!). I was also terribly proud of my beautifully engineered object oriented framework that would allow me to add so many more features that I was sure he was going to ask for.
The let down
Guess what, after six months of hard work he didn’t like a single new feature and was upset about several missing features from his original software. Those features were never captured in the requirements document. I was upset, how inconsiderate of him to not respect the requirements and to not shower me for praise for all of the great features I added. Of course this was defensive thinking, I was actually mad at myself for having let him down.
I went away and reflected upon the golden rule of customer service which is “the customer is always right”. In this case the customer was my boss, but this line of thinking meant somewhere along the line I must have screwed up! How could that be? I had done everything by the software engineering book.
The realization
After thinking about it at some length (some would call it obsessing) I realized that the problem wasn’t me and the problem wasn’t the customer, it was just that the software engineering principles I learned in school had overlooked basic human nature. People don’t really know exactly what they want until they try it, and their needs and desires constantly change. In other words I had learned a truth about software development:
Requirements are constantly changing.
A new set of principles
From that day forward I vowed to give my customers working software as early as possible and incrementally improve it according to their feedback. This meant: no more grandiose software frameworks in anticipation of feature requests that never came, no surprise features that were never asked for, and no more withholding software from the customer until it was “just right”.
Final words
These principles have of served me well for over 15 years and I continue to see them echoed in articles about software development best practices, and often more eloquently than I have expressed them here. Unfortunately the core ideas are still somewhat misunderstood and overlooked in many larger organizations that declare themselves as having adopted "agile" engineering principles.
Having shared my personal experience, perhaps Agile development principles might make a bit more sense to you. Always remember: being agile is not about scrum meetings, velocity calculations, planning poker, or any of those process artefacts. It is about putting the ego aside and collaborating directly with the customer.
Acknowledgements
Thanks for taking the time to read this personal account of my early days as a software developer, I hope you enjoyed it. I also want to give a big thank you to my old boss Dr. Matthew Shapiro who inspired in me a love of software development and who was always so patient, supportive, and inspirational.
Recommended reading
Here are some related articles that I recommend reading:
And if you really liked this article check out my other articles at CodeProject.com.
