Common Sense Software Engineering – Part III; Risk Analysis

The following piece describes a process for performing “Risk Analysis”, also known as “Risk Management”. What the reader will find is that contrary to popular development paradigms, true software engineering practices require quite a bit of upfront analysis for new project development as the prior piece on “Requirements Analysis” demonstrated.

In the frenzy of the so called “new development environments”, many technical managers as well as professional developers have attempted and still are attempting to find techniques that will allow them to avoid such in-depth processes and yet still create quality software deliverables. No matter how much marketing, PR, and other technical propaganda is thrown over the issue of quality analysis, without it, quality will never be part of the end result.

On another note, the reader will note that Steve McConnell and other software engineering analysts of years ago are relied upon for this work as has been the same with the other articles. Steve McConnell’s 1996 classic, “Rapid Development”, to this day has never been refuted and is still in fact being corroborated by subsequent studies in this arena. As a result, for many quality business technical personnel, it is still considered the “Bible” of software engineering.

>>>

One of the most important aspects of the management of any software project, large or small, is the management of risk. “Risk” in terms of project management, is any situation that either prevents a project from reaching a successful and timely conclusion or interferes to delay a well-run project towards that same conclusion.

Overwhelmingly, IT project management ignores this very crucial aspect of software development. As a result, project development schedules are affected negatively with the same managers putting pressure on their staff to make up for lost time, when it should have been properly planned for.

Every software project is faced with risks of all types and severities. In fact, any endeavor is faced with similar potential obstacles to completion. If you attempt to climb a mountain you risk the possibility of breaking a limb and in the more treacherous climbs, your life.

Even simply commuting or driving to work in the morning presents commuters with any number of potential risks from simple schedule delays to derailment. Though such situations tend to have low levels of occurrence they do in fact happen.

Every software project when initiated always begins with one primary risk to timely completion; that of technical support for setting up development resources when needed as well as moving the completed project to production. Though many companies have extensive technical support infrastructures, their response to development requirements are always measured against the needs of the production environment making all such requests difficult to measure against for scheduling constraints. Production support is somewhat better. However, here too, like development, it is also dependent on who is assigned to your task and their experience level.

By no means does this mean that every project will experience technical support delays but it can happen and must be planned for accordingly.

Planning for project risk usually falls into one of five categories with the most credible and most valuable finding itself relegated to the “bottom of the heap”:

• Crisis Management

       “Firefighting”; addresses risks after only they have become problems

• Fix on Failure

       Detect and react to risks quickly but only after they have occurred

• Risk Mitigation

      Plan ahead of time to provide resources to cover risks if they occur,

      but do nothing to eliminate them initially

• Prevention

     Implement and execute a plan as part of the software project to identify

     risks and prevent them from becoming problems

• Elimination of Root Causes

      Identify and eliminate factors that make it possible for risks to exist at all

Planning for the elimination of potential risk factors is by far the best cost and time saving process a software project manager can possibly infuse into his or her planning of a project. To ignore this most valuable asset for any other level of risk management is simply to increase the “gambling” quotient against a project. Here, a project manager begins to enter the “wishful thinking” syndrome as he or she gambles against the chance that a risk factor will not crop up that can’t be handled properly.

Risk management, outside of schedule estimation, is the most complex issue facing software project managers. Risk management, also known as “risk assessment”, requires quite a bit of up front effort prior to the beginning of any technical aspects of a project. As such, it is an effort that most managers would prefer to avoid and the still high numbers of project failure in the industry demonstrate this.

This effort can then be broken down into the following components which the following chart demonstrates:

(Rapid Application, Steve McConnell – 1996)

Click Image to Enlarge

The most common project risks also fall into the “classic mistakes” category and can be summed up in the following list:

•    Feature creep
•    Requirements or developer gold-plating
•    Shortchanged quality
•    Overly optimistic schedules
•    Inadequate design
•    Silver-bullet syndrome
•    Research oriented development
•    Weak personnel
•    Contractor failure
•    Friction between developers and users\customers

Risks to a successful project life-cycle can come in all different sizes and shapes making software development one of the most risk-adverse technical endeavors. Some observers who have studied the field believe that gambling in a Las Vegas casino offers better odds at success than those currently proffered for the completion of a project successfully.

As a result, any project that has no risk management planning implemented is guaranteed to fail and those that don’t would have simply succeeded by sheer luck. However, it is surprising to find out just how many project managers who have succeeded with a few small projects tend to use them as their yardstick when planning more complex and difficult projects.

It is also just as surprising to see technicians come out of very difficult development environments convinced that the arduousness of the scheduling and the monumental stress are actually part and parcel to successful development stratagems when instead such environments are simply examples of poor planning, poor scheduling practices, and poor management.

Though not listed in the preceding group, stress is actually a terrible risk to quality software development and has been found to be a major factor in high rates of defects; sometimes up to 40 percent of errors are due to the stress levels placed on developers.

Even so, many managers and even developers have come to believe that the “risk-factor” of stress is simply part and parcel to the software industry and so much so it is not even considered as a risk-factor when planning projects. Since management likes to portray high stress levels with a reduction in costs and development time many technicians have come to believe that without such working conditions projects cannot be completed on time. However, just the opposite has been found; stress instead increases cost and the time it takes to complete a given project and for a variety of reasons. For example, high levels of stress reduce developer creativity, lowers morale, lowers the ability to concentrate, and simply fatigues technicians to a point where they cannot produce optimally (see chapter 9.1, “Overly Optimistic Scheduling”, “rapid Development, Steve McConnell – 1996).

Most such stress comes from schedules determined by management without any understanding of what they are asking. And again we can turn to the financial organization in the trading room where we often find stress levels that are so enormous that most often technicians leave after around 12 to 18 months. High turnover, as a result, drives up the costs to such companies and their project development dramatically but few give it any thought and in fact even plan it into their budgets.

Despite the reputations of the quality of the technicians that work for American financial organizations, such organizations that do engage development with high levels of stress do not turn out high quality products because they simply can’t.

Excessive or irrational schedules are probably the single most destructive influence in all of software (Jones – 1991, 1994).

To get an idea just how many risk factors are involved in software development beyond the few already mentioned, please take a look at the detailed risk-factor chart that follows …

Potential project risks that will affect schedule…

Estimating Risk Exposure

Estimating risk exposure is a rather subjective form of analysis but nonetheless must be performed in order to be able to understand the severity of risk factors to a project. However, even subjectivity can be made more accurate by the use of basic risk exposure analysis.

Risk exposure analysis is comprised of two estimations; that of the size of potential loss (in time) from an identified risk-factor and the corresponding probability that the loss will actually occur. A simple formula is standard in the industry for determining a risk-exposure calculation:

(% probability of loss) * (size of loss in weeks) = risk exposure factor

For example, if we were to use any development project at one major financial organization that the author worked at a number of years ago as a baseline, we would have to assume that there is upwards of a 75% probability of some impact on a project by delays from technical support. If we assume a conservative estimate of a loss in time of approximately 4 weeks than the risk exposure would be as follows:

.75 * 4 = 3 (weeks)

The result is a risk exposure factor of a possible loss of 3 weeks. Since we are only estimating a 75% probability of this occurring we are then not expecting to lose the complete 4 weeks in time.

Estimating the size of a loss is much easier than doing the same for the probability of a loss. As a result, there are several possible activities that can be performed to more accurately determine this part of the equation.

• Have the person most familiar with the system as well as the development environment and political infrastructure estimate the probability of each risk and then convene a risk-estimate review.
• Use the Delphi approach where each project team-member estimates each risk-factor individually. Then convene risk-estimate reviews to discuss and determine the most likely probability of each risk-factor until the entire team is satisfied with a final analysis.
• Use simple “betting analogies” with personally significant amounts of money. For example, “If the facilities are ready on time you would win $125.00, if they are not you would lose $100.00. The risk probability then would be the dollar amount on the downside divided by the total dollar amount at stake ( 100.00 / (100.00 +$125.00) = .44 ). (paraphrased from “Rapid Development” Steve McConnell – 1996)
• Use “adjective calibration” where each team member would select a risk level in terms of a verbal scale of phrases from “highly likely” through “highly unlikely”. Then convert the verbal assessments to quantitative assessments (Boehm – 1989).

These methodologies may look rather unscientific. However, such estimation initially has no basis in fact so such a process would be only a “best estimate” at the time this is accomplished. Estimating the effort and time for a complete project life-cycle is theoretically impossible at initiation unless there is a good amount of metrics available from previous projects that allow a new project to be compared with the measurements of a similar project that has already been accomplished.

Once a list of risk factors have been determined along with their loss-probabilities, loss sizes in weeks, and risk exposure (RE) percentages, a sorted listing based upon the RE should be laid out for planning purposes. A sample of such a listing can be found below…

Example of a Prioritized Risk-Assessment Table

Setting up the table in the above sorted manner produces a listing of risks that are ordered in terms of exposure, the greatest amount of exposure topping the list. If the top five risks in the above table were to be planned for successfully, 9.8 weeks of scheduled overruns could be eliminated.

The table above also provides information on those risks that could be considered low priority as a result of their minimal chance of occurrence. By documenting such information in such a way then, time wasted on planning for low-exposure risks is also eliminated.

It should be noted that “risk assessment as the chart above demonstrates, is only a subjective estimation of the possibilities that can affect the outcome of a project. Because such an assessment is performed in a subjective manner the accuracy is completely dependent on the quality of the input given to it. The more effort on both the part of the project manager as well as the team given to this process, the more likely the resulting assessments will be within the assigned time ranges thus providing a clearer understanding as to how much effort could be saved by proper planning for such project delays. However, it cannot be stressed enough that no matter how well thought out and performed, project risk assessment will always remain a “best estimate” scenario.

The question then arises after such a process has been completed is how one goes about managing such risk factors. If we were to apply risk resolution to the top 10 risk factors (see page 32) that could affect any software development project then the common approaches and recommendations for controlling such problems can be found in the table below.

Means of Controlling The Most Common Schedule Risks