The Standalone Programmer: A question of quality

Revisions

January 3rd, 2003

1. Corrected problem with various links
2. Added "Initialize variables" to code-quality guidelines
3. Added "Understand and use const" to code-quality guidelines
4. Added "Understand the difference between a reference and a pointer" to code-quality guidelines
5. Added "Traces are great" to code-quality guidelines
6. Added a section called "Code that express intent"
7. Added paragraphs to the "Quality of the end product" section

January 4th, 2003

1. Added a "RAII" section to the code quality guidelines section
2. Added a link to a performance test bed in the "The cost of quality"

Preface

This article is actually the original document I was working on when I published my earlier article "The Standalone Programmer: Delivering high quality results".  I have decided not to delete the earlier article and just leave the duplications intact.  I apologize if this is confusing. 

Also, thank you to all people who responded earlier to this article (and my others) and provided great feedback.

Introduction

I have been reading numerous books, magazines and web site articles on the subject of software quality. All of these sources point to the need for detailed processes and staff to design, test and retest software to insure high quality results. This presumption that it takes teams to develop quality software cannot be entirely false? Can it? Surely there is hope for the standalone developer to develop quality software. Unfortunately, the software industry as a whole appears to argue against this possibility.

To determine the truth of this assumption, we need to first determine a meaningful and measurable definition of �high quality software�. Part of the problem that I see is that no one defines �high quality software� the same. Everyone has their own standard of quality and not many people agree on the practicalities of implementing �high quality software�. The definitions that I have come across are usually quite vague and not measurable. The definition also appears to differ depending on who is asked the question.

In this article I want to attempt to answer the questions �Can standalone programmers develop high quality software? And if so, How?�

What is high quality software?

Ask a newbie programmer and his definition of high quality software might be software which does not crash. Ask a bean counter and they may say it is �bug free� (what does that really mean anyway?). Ask a maintenance programmer and they will say that high quality software is code which can be easily understood, debugged and updated. Ask someone flying the space shuttle and they will tell you high quality software is software that always works right all the time. Ask a software project manager he/she might say that high quality software is software that does what it is supposed to do flawlessly. Ask an end user and they might say that high quality software does not crash and does what they want it to do when they want it to do it and as quickly as they want it done.

To make matters worse, the definition is often different based on what product is being developed. Software that can cause death such as medical support software must be functionally bug free and fault tolerant. Software to run a typical in-house order entry application needs to be feature complete and reasonably stable, but may not need to be fault tolerant at all. Software for a commercial web e-commerce site may need to be fault tolerant and visually bug free.

Even if we manage to nail down a definition for high quality software, the word �bug� keeps cropping up. What does �bug� mean? Is a bug a missing or incomplete feature, something that causes a crash, misspelled text, poor performance, etc? Who decides what is a bug and what is not a bug?

Obviously, the definition of high quality software up to this point is more an art than a science. However, one fact is always the same: High quality is always important regardless of the meaning. What is needed is not a one-size-fits-all definition but a more generic definition that is refined on a project-by-project (or even component-by-component) basis. From this perspective it becomes clear that what is needed is a plan for every project and every component within a project.

Planning for quality

A plan! What is that? Undoubtedly, many of us have �gotten by� without ever writing an architectural design or detailed design document. As a standalone programmer this is easy to do because who is going to see it anyway, right? Besides design documents aren�t our job, right? We�re just hired guns, right? From the perspective of truly high quality software, though, this is the completely wrong approach. High quality software demands definition not just of the code but of the requirements and expectations of the code. If there is no definition, how can we possibly measure the quality of the software we develop?

Most (if not all) large software firms such as Microsoft, IBM, SUN have some form of formal design and documentation process that precedes the actual coding of the software. This process typically involves modeling the project in OO terms sometimes using UML or other tools. It also involves listing the functional requirements of the project and often specifies in great detail performance metrics for the various components of the project.

How can the standalone programmer compete with this? Obviously we cannot invest the amount of time and resources into design as large corporations can. Fortunately, we are not usually called upon to develop software quite as complex as MS Office. However, the software we do develop can be more critical to our employers or clients (or jobs/careers) than MS Office. MS Office rarely is a direct component of the corporate bottom line, but the software we develop often is.

The solution is to scale down the requirements gathering, analysis, design and documentation requirements to reasonable levels for our needs. Instead of treating each stage of design (requirements analysis, architectural design, etc) as separate pieces it is often advantageous to treat them as a single step I call pre-coding.

Once we know the requirements, we need to make two more critical. We must make a decision as to what high quality means for THIS PROJECT. Once we have made that decision, we must then decide how we can measure quality for THIS PROJECT. The method of measuring quality must be as quantitative as possible.

I am not going to go into detail on how to write a pre-coding document because I am not sure that there is any one �right way� or if there is, that I know it. The critical point is to answer all of the important pre-coding questions and define a metric for measuring quality. Beyond that, I don�t think it matters if you use MS Word, Visio, Rational Rose, etc.

Quality of the end product

One of the pitfalls that many software developers fall into is believing that high quality code == high quality software. This is not always the case and in fact is often not the case at all. Just because code is well written does not mean that the end product will be of high quality. Having a definition of high quality for a given product should reveal where the emphasis should be. In many cases, the user experience will be paramount so a speedy and well-designed UI is of the utmost importance. For other applications, fault tolerance and error recovery may be of critical importance while the UI needs to be only minimal.

Regardless of where the emphasis is placed, quality software does not begin with the code it begins with an understanding of the expectations and final uses of the software. The code is intended to support those ends.

Probably the most misunderstood aspect of quality software is understanding the needs and expectations of the end users and incorporating those needs and expectations into the software at an early stage and at every stage. For E-commerce web site front ends, the end user will be a customer, for in-house order entry applications. For an e-commerce web site back end, you will be the end user and you need to think about that that means as well.

I strongly suggest that you not only consider the end-users in your design and development process, but that you make that involvement a format part of the process.  Dedicate time and effort to it because it can be the only way to measure quality when user-perception is paramount.  If the definition of quality for a give project is "Suzy Q in Cubicle A13 must like it", then you need to have Suzy Q involved from the earliest possible moment.  (you may find my earlier articles on usability testing and the myth of efficiency useful.)

This brings up another important point.  This article is trying to make a subjective word "quality" have a quantitative value.  This is entirely possible so long as we do not forget the subjective nature that got us here to start with.  Never forget that your perception of quality may not be what the customers or end-users perception is.  Be prepared to alter your behavior and practices to better meet the customers expectations because ultimately they are the one paying the bill.  This does not mean that important areas of design and code should be sacrificed, but they need to be considered in the light of the customers needs and expectations.

Bugs to keep and bugs to kill

Depending on the definition of high quality that you select for a given project, you must decide which bugs require immediate correction and which should be logged and left for later. I don�t believe that it is possible to develop a non-trivial application and have no bugs. I also don�t believe that it is necessarily the right thing to try and eliminate every known bug.

You may be thinking that I am crazy, but I am not alone in this belief. Many highly regarded authors and professionals agree with me on this point. As a standalone developer it is unlikely that you have the resources to develop software on-time, on-budget, of high quality and bug free. We must make good decisions as to the bugs we take time to eliminate and the ones we leave. The ones we eliminate are the ones which affect the quality of the end product according to the definition of quality we selected.

Obviously, a bug that prevents the program from executing its intended function under normal circumstances will have to be dealt with, but a problem with the layout of a dialog or a problem with a specific function which is rarely used may need to be ignored at least for the first release. The key to remember is that there are features of the design that are more important that some bugs. The need for excellent logging in a back-end server application may be more important than fixing some bug in a completely non-critical part of the application.

High quality code

Regardless of the standard for high-quality code that you adopt for a give project, there are certain standard code quality issues that should always be considered. I have struggled with many basic rules of high-quality software and still do. I sometimes find it tedious to write adequate code comments, I often fail to check return codes, I often start writing code before I adequately design it. Most of us know what these basic rules of �good� coding are but I am going to list some of them here anyway.

1. Commenting the code.

• Someone (not sure who) wrote �If you can�t say it in code, say it in comments.� If code is unclear, complex or just plain long, write a comment to explain why it is the way it is.
• Keep it simple. A good comment can be just a few words or whole paragraphs. The key is to keep it simple and comment the important decisions you made that affected the code and the �gotchas� a maintenance programmer should be aware of.
• Keep in mind that maintenance programmers (including yourself) rarely maintain comments. They maintain code. So, if you write a comment like �// a = b� it is likely that someday the code will change to a = c, but the comment will remain �// a = b�. So, don�t comment the obvious.
  

2. Validating function inputs

• It is wonderful to be debugging code and get an assert dialog or trace output which tells me what went wrong before the app goes BOOM.
• Learn to use the IsBadReadPtr and IsBadWritePtr when working with C++ on Windows.  (This article may be useful.)
• Validate in release builds too. Ultimately it is the release build that we want to prevent critical failures in, right? Obviously performance is always a consideration, but I would venture a guess that 95-98% of the code which needs input validation will not be adversely affected by it.
  

3. Checking function returns

• Know what return values to expect and why they happen.
• In the code, deal with each return value appropriately.
• Deal with the unexpected return values. If a function can return 1, 2, 3, or 4 plan for it to return some other codes. Therefore, if you are using a switch statement, include a default branch to deal with the unexpected return values.
  

4. Initialize variables

• When possible, initialize variables to actual working values, not dummy values (ie. instead of int x = 0; x = GetSomeValue();, write int x = GetSomeValue();_
• When working values are not available, initialize variables to zero, null or some other value.
• This can be especially frustrating when performing bit-wise operations. DWORD dwSomeValue; dwSomeValue |= WS_SOMEFLAG; will yield unpredictable results.
• When you dereference a pointer, set its value to NULL or some other meaningless value like 0x0badf00d.
  

5. Understand and use const and const references

• If you do not intend to alter the value of a functions input parameter declare the input parameter as const
• When passing objects (such as CString) as function input parameters AND you do not intend to alter their values, declare the input parameter as a const reference const& CString
• If a method of a class does not alter the "state" of the class (such as GetSomeValue), declare the method as const so that the compiler can optimize its output better and other programmers have an idea of what to expect
• When possible, put const values on the left side of comparison expressions (ie. instead of if (x == 1)..., use if (1 == x)). This works with all const values, not just literals.
  

6. Understand the difference between a reference and a pointer

• References are excellent tools for making code more readable and reliable
• References can often be used in-place of pointers and provide a "safer" and cleaner approach
• References also express intent as to what you intend to do with a value so use it when possible
• Caution: Don't be afraid to use pointers when a pointer is best.
  

7. Catch exceptions at appropriate locations

• Know which functions might throw an exception and deal with the exceptions appropriately
• Failing to deal with exceptions can cause memory leaks
• If you don�t catch it you loose control of the system
  

8. Use dog tags

• This is especially good for server-side back-end applications than must be up 24/7.
• Critical data areas can be cross-checked by prepending and appending small data areas to them and initializing those data areas to pre-determined values.
• Before working with the class/structure, check that the dog tags contain the expected values. If they do not, you know that there is a problem.
• See this article for more helpful information. http://www.codeproject.com/debug/debug_macros.asp
  

9. assert is wonderful

• The assert macros is arguably the most powerful feature in the C++ language.
• (From �Game Development Gems�). In C and C++, constant strings embedded in code are treated like pointers. Therefore the code �assert(bSomeFlag && �bSomeFlag must be set to TRUE before bSomeOtherFlag can be set�);� will work just as well as �assert(bSomeFlag)�, but will provide more information to the person doing the debugging.
  

10. Traces are great

• Traces provide the programmer the opportunity to report-back details without interrupting program flow.
• Traces can be used in release builds when needed
• When using traces in debug builds especially, consider providing more information in traces even if performance takes a hit
  

11. RAII - Resource Acquisition Is Initialization

• A big name for a simple concept.
• Objects should take responsibility for allocating and freeing resources on construction/destruction
• When an object is destroyed it is expected to free its resources
• In C++ this is quite easy to accomplish by proper use of class constructors and destructors. Deterministic finalization helps alot.
  

12. Coping with error conditions

• I have struggled to find a reason why detailed designs are needed, but this is definitely the one that makes the most sense.
• Design functions and classes to not just deal with error conditions but adequately report the error condition to the calling class/function/module/etc. Functions with no return values ( void SomeFunction() ) and no way to report errors should be carefully reviewed. How will the report errors? With exceptions and if so, what exceptions will they throw? If not, how can the function be modified to report error conditions.
• Is a given error condition worthy of being logged for later review?
  

13. Use transactions when updating databases

• I can�t stress this enough. If you are working with databases and are doing updates (which most applications do), you need to be using transactions to insure that your operations are committed correctly.
• When updating a DB, it is critical to check return codes from each update statement or stored procedure and only COMMIT the transaction if ALL updates were successful.
• This reinforces the fact that function calls which perform database updates MUST have a designed means for reporting errors.
  

14. Learn about and use design patterns

• Reinventing the wheel is not a lost art, but should be.
• If you decide to use a particular design pattern in your code, comment your choice so that you and future programmers can be informed and hopefully maintain the design patterns implementation
  

15. Think of code as a liability, not an asset

• More code is not always better.
• Reuse code whenever possible through derivation, consolidation, delegation or whatever.
• Every line of code you write has the potential for bugs and change
  

16. Naming conventions

• This issue has been beaten to death but deserves a mention.
• Give meaningful names to all functions, classes, structs, variables, constants, macros, etc.
• About the only thing that gets meaningless names are simple iterators (ie. It, x, y, I) used for basic looping.
• Try and standardize across all projects you work on.
  

Self debugging code

There are no (or at least very few) totally bug free applications out there that are non-trivial in nature. And even if an application is bug free, that does not mean that it will not crash as a result of a full disk or other problem. The problem with debugging applications is that it takes a lot of time. In fact, according to some sources, it can take as much time to debug an application as the actual code writing takes. And this only accounts for the debugging time spent during the development cycle. What happens after a product is released to the end-users? Problems will undoubtedly arise. Accessing the quality of software that has been released can be a very difficult task and will typically only be an issue when things go wrong. After all, who worries about a light bulb that�s not burned out?

This past week, one of the critical software systems at my current employer crashed unexpectedly at a most inopportune time. My first step was to review the log that the program produces to determine if the cause of the problem had been logged. Sure enough, the log contained an Error message along with a brief description of the problem. Also, the log contained the source code file name and line # where the error occurred and the name of the function which trapped the error. It was an access violation (GASP). I was able to review the code and correct the problem in less than 45 minutes thanks to this information. It turns out that I was overwriting an array.

The moral to this story, though, is that the application (which happens to be a windows NT Service that drives a web site) basically debugged itself. Without the detailed error log, I could have spent hours or days tracking down the problem while a critical business function remained offline. As you have probably already guessed, the log is the result of previous sleepless nights not the product of some personal genius. Been there, done that, don�t want to do it again.

Self-debugging code comes in 2 complementary forms. The first form of self-debugging code applies to all code regardless of whether it is intended for a GUI, console app, web app, etc. This form mostly benefits the developer during the product development cycle but offers little once the product has been shipped. The second form of self-debugging code extends the debugging qualities to the released product through self-reported errors, logs, stack traces, etc.

The first form involves applying the techniques of validating function inputs and checking return values. These steps can and should be applied to all code regardless of how critical the code is or where it will be used. When debugging applications which use the assert macro (or similar construct) liberally and validate all function inputs, many bugs are detected immediately and require little time to correct because they were anticipated previously. The additional step of commenting helps to debug problems that asserts don�t automatically detect by helping the developer recall how and why an algorithm is implemented.

The second form involves extending the first form to include actively logging application function information, error conditions, catching exceptions (even ones which can�t safely be handled) and providing descriptions for important internal functions and data that can be logged when errors occur. At its simplest form this involves logging critical error conditions such as exceptions. The log should contain enough information to identify where in the code to begin looking. Additionally, details about the data being worked on before the crash can be useful and aide in debugging. Some ideas for this form of self-debugging code include:

• Maintain a file-based log using a commit-on-write strategy
• Log all error conditions caught through exceptions, invalid input parameters or unexpected function return values
• Log entry and exit points to critical functions which don�t occur often
• Log the name of the code file (__FILE__) and the line of code (__LINE__) where the log entry was made along with the name of the function.
• Log the date/time of the condition.
• Log any details you have regarding the error itself. (ie. Type of exception, etc.)
• If possible, log a stack trace.
• Log any details you have regarding the data the app was working on when the problem occurred.
  • a. WARNING! Obvoiusly, the data may be the source of the problem or may have been damaged as a result of the problem so be careful in how you report this information

This second form of self-debugging code is often not needed and is overboard for many applications. It is however, quite useful for in-house applications and pretty much any server-side application which has to run 24/7.

One technique that I have found useful in server-oriented applications is to utilize a custom handle mechanism when working with data. Instead of passing data around as pointers, pass the handles to the data except where performance is absolutely critical. When the handle is assigned to the pointer, assign additional descriptive data to the handle to identify the data. Then, if an error occurs involving the data, instead of trusting that the data or the pointer is valid, retrieve the description from the handle and log that information.

Code that expresses intent

What is he talking about?  That's probably what you are thinking about now.  I am not really referring to the concept of "Intentional Software" but it's not too far off.  What I am referring to is how code in its syntax, structure, format and naming conventions can assist maintenance programmers, integrators and yourself by being as explicit as possible. Whenever possible (as much as the language allows) programmers should strive to be explicit.  This can be achieved in very simple ways by following good code-quality guidelines and being consistent.  One caveat to this is that although comments represent one means for expressing intent they are the most volatile because they are the least likely to be maintained over time and programmers often overlook them for this very reason.  Comments should be used to expound on the intent that the code already expresses, not provide intent to code that exposes none.

So, what are some practical ways of achieving this beyond the obvious? Well, here is my list of a few simple and powerful techniques for achieving this: (much of this is duplicated from the code-quality guidelines above. I duplicated it because I believe it deserves more emphasis.)

1. Use well formed variable names (method input parameters) that tell you and others what the variable is used for, its type and when possible its lifetime

2. Use thoughtful and meaningful names for classes and functions.

3. Use const when possible to express the intent of the code NOT to alter a value

4. Use const to express the intent that a class method will not alter its state

5. Use enum's (or at least const or #define's) for return values when possible to express the intent that more than one value can be returned and what should be expected

6. Encapsulate decisions based on the state of a class so that the intent of those decisions is visible through the code and interface

7. Avoid modality in your classes when possible. Instead when modes are needed, consider developing specialized classes for those modes which can better express the intent of that code.

8. (This goes with #7). Classes (as much as possible) should fulfill a single purpose that is expressed clearly through the classes name, its interface and code. 

9. Provide a structured means for returning errors, warnings, etc from functions. If a simple return type or enum is not enough, consider passing in an error structure (see CFile::Open). This expresses the intent of how errors will be dealt with and what callers of that function can expect. If exceptions are your choice, take the time to write a custom exception class which expresses the intent on what error information to expect.

The cost of quality

If you're like me you've probably read all of this and are now thinking that implementing some of these concepts and guidelines is going to take allot of extra time and is going to adversely affect the performance of your application as well as your schedules.  The reality is that there is always a price to pay for quality.  How big a price depends on many factors and as a software designer we must make decisions which will have an impact in various ways.

I believe (and have experienced) that by planning for quality and maintaining a continuous commitment to it in everything I do, I can greatly reduce the time cost of using best practices.  The issue of performance, however, is bound to come up and rightfully so.  Part of the quality of an application will often be measured by its performance.  I believe that most code-quality guidelines will not adversely affect performance when used appropriately,  However, in the past I have not had any hard and fast performance numbers to measure the cost of various code-quality guidelines (input validation, IsBadReadPtr, IsBadWritePtr, range checking, dog tags, etc), so I have decided to develop a test bed project and write an article on my findings.  This article can be found at http://www.codeproject.com/useritems/StandaloneProgrammer4.asp.

12 steps to quality software

I want to talk a little about something that Colin Davis pointed me to. A man named Dr. W. Edwards Deming listed 14 steps to quality management that I found quite interesting. While his steps are not targeted at software development, they clearly can apply in many cases. (For more info on Deming, search google or follow this link http://www.murtongroup.com/documentation/deming1.htm). What I would like to do is to rewrite his steps so that they target software development. Below is my rewrite of the steps. (Please note that much of the text of these steps was taken directly from the link listed above and is not original to myself.)

1. Maintain a consistent Quality message and purpose towards improvement

• Always remember that high quality is critical to successful software projects.
• When problems arise, take action. Send a message that quality is important to you.
• Remember that high quality is as much a process as it is a product.
• Let other people know that you will go the extra mile for quality.

2. Quality requires a commitment to continuously improve.

• There is always room for improvement in design, implementation, deployment, schedule, user interaction, etc.
• Testing is critical to quality
• Be open to new ideas, technologies, techniques
• Empower yourself for improvement.
  • Start using a bug tracker
  • Learn to use modeling tools
  • Study other development techniques (such as extreme programming, etc).

3. Switch to defect prevention approach rather then the defect detection.

• Look for problems before they happen.
  • Validate input parameters to functions
  • Check return values
• Learn what mistakes you make the most and find ways to cope with them
• Use a bug tracker to keep track of past problems and review the bug report frequently (even the list of fixed bugs)
• Fix bugs early in the process

4. Find and use components that are designed with quality, reliability of delivery and commitment to improve.

• Third party components must be chosen based on their level of quality not on any other factor such as �coolness�, �looks good�, etc.
• When possible, try and work with those people who have the same commitment to quality

5. Constantly improve the production system or service to reduce quality issues.

• Improve your development environment to remove barriers to production
• Learn to manage your time better
• Setup a build process (daily builds can be GREAT)
• Delegate non-quality related work to others when possible to free yourself to focus on quality issues
• Find and use programming aids
  • WndTabs
  • VisualAssist
  • Etc.

6. Understand the concepts of variation, improvement and total quality approach.

• Realize that changes are inevitable but don�t have to be bad.
• When writing code, remember that any given function, method, class has a high probability of requiring future changes. Architect systems (through OO techniques) with this in mind.
• Make total quality part of each project

7. Establish an environment for improvement and dispel any fears of change.

• Be prepared for changes in your own views, technologies, languages, styles, etc.
• Find time to test new ideas that might improve quality

8. Change Supervision to coaching and support from chasing and a blame culture.

• Evangelize your boss, coworkers so that they become support for what you do.
• Empower others to perform as many tasks as reasonably possible

9. Work to remove organizational barriers to Quality Improvement.

• Again, evangelize those around you.
• Try and educate people on how quality improvement can be achieved.
• Be prepared to work with others to find solutions that work for your organization.
• Watch for problems caused by office politics, etc and try and find ways to overcome them

10. Discard unrealistic targets and unnecessary pre-dispositions.

• Learn to plan and schedule better. Most software developers are terrible at making schedules.
• Eliminate unnecessary predispositions that hinder quality improvement. An example is �I hate VB.� If VB can be used to create higher quality software (through prototyping, test beds, demonstrations of what not to do, whatever), then use it.
• Keep an open mind, but be cautious about jumping into new ideas, technologies and techniques

11. Take pride in your work.

• Quality is more important when there is personal image at stake.
• An old clich�: �A job worth doing is a job worth doing well�.

12. Train and educate for self-improvement.

• Move beyond reading only technology books (and articles, magazines, etc) to reading materials that can help you improve quality.
• Keep up-to-date with the latest trends and developments in the industry.

Conclusion

So, is it possible for a standalone programmer to develop high quality code? The answer is YES with caution. The caution comes from the realization that developing high quality code will mean working smarter, doing pre-coding work and developing better work habits. All of the requirements of high quality software are not self existing in the standalone developers world. Most of us do not have highly structured and detailed requirements documents, most do not have enough control over delivery schedules or feature sets. These issues can frustrate quality.

With a commitment to improvement and quality, a good work environment, a willingness to learn and improve oneself and a little time, I believe that standalone programmers can not only compete with teams of programmers but also beat them in many aspects. By being focused on their individual needs and not caught into team politics and problems, individual developers can focus on the end product more and have more control over the decisions made regarding their projects.