Coding with Klocwork: A Dev Perspective

Introduction

Application security and code quality. Two things we focus a lot of energy on, but two things that are also getting more difficult as application complexity and the number of application components are increasing.

This is where code refactoring, analysis, and security tools are helpful. Recently I had the privilege of testing a controlled release of Klocwork static code analysis 10.2. A code refactoring, component security and reporting platform. This post is about my thoughts on this tool.

First a little background. I am a client side .NET developer turned PHP web app developer. Most of what I write are data parsing, and file analysis tools in the B2B space. I’ve moved from heads-down coding, into coding as a hobby, and now a DevOps market analyst.

I call myself a duct tape coder, because I’m good at prototyping, but not great at production code. Even more of a reason to have code analysis in my back pocket. My perspective is more from a development manager point of view than a heads-down developer POV.

Environment

In my testing, I had two test scenarios and then tested some one-off features as well. They were:

1. A win32 application for analyzing file shares and displaying the results in the browser. Code here.
2. And an e-commerce web application for a winery.

My environment was a Windows 2012 Server (16GB RAM, Dual-Core Xeon processor) on CloudShare using an Apache web server. I used Mono for running .NET applications on Apache (normally you would not do this, but I was trying to keep everything on one machine) and my IDE was Visual Studio 2012. I originally started with Visual Studio 2015 because I wanted to see if it worked, but not yet.

In a real-life scenario there would be a developer box for static code analysis and a separate web server for the Klocwork web application. You would deploy from the dev box to a web server and analysis would happen on the released application, but likely on the dev box as well. What is nice is you can do this for all your environment’s dev, test, integration, staging, and prod.

Install

Of course my testing started with the install. The install experience is very important to me. My expectations are that I should be able to get started within 30 minutes with any new tool. With Klocwork, I was initially concerned it would be complex after I received a license file and download links. But it was not, just a 15-minute install (mind you I am not a server expert). Here are some of my notes on the install:

• It was a two-part install. One for the server and one for the desktop static code analysis tools. In my case Visual Studio, but it also supports Eclipse.
• I found it very cool that they have the whole stack for the server install (Klocwork, Apache, FLEXlm, MySQL). I had already installed Bitnami WAMP stack and thought it would be an issue, but it wasn’t. I did have to set Klocwork to run on a separate port than 8080.
• Install was fast, I expected to do a lot of web server config. The only gotcha was that the Klocwork license server service did not start automatically. This prevented me from doing my initial login. I started the service manually and it worked just fine.

Installed and time to start analyzing! I created a Klocwork project for each of my VS projects. My projects were fairly small. Only about 100 files or so in the largest. The next step was to run my initial full analysis on both of my project file. Incremental analysis happens after this with each build.

I completed the run of both my applications. It went well once I learned the commands. There were three commands per project. There were a few errors, but they did not seem to impact anything I was looking at. I then reviewed all the results in the Klocwork web app.

It should be noted that my tests were very individualistic. However the team and reporting aspects I think are the most important value, as I will explain below.

In the application I spent a lot of time on the code analysis results and the reporting. Here are the pros, the cons and my wish list from my testing:

Cons

• There are a few annoying things with full analysis runs. First of all its command line. Second you want your projects in a convenient location like C:\ for easier commands, but this is not best practice. There are a lot of steps to the first run that I think they could be consolidated. My inclination would be to write an application/bat file that allowed me to browse to an SLN file and run the analysis.
• I wish that the Visual Studio plugin was available in the VS gallery. I think in future versions of VS they will be required to do this.
• I felt like some parts of the UX were a bit dated.
• C# support is limited to around 40 checkers compared to the 250 or so for C++. And I really wanted the on-the-fly (code completion) analysis in C#. It’s too easy to forget a run.
• A better alerting system is needed. It’s possible to setup alerting, but it is complex. What is available is email alerts, but you have to set up special rules so Klocwork knows who the alerts go to. You have to work with a source ownership file or build logs. I also would want to see it even expanded to texting, Zapier etc. Or integration with alert tools like PagerDuty or VictorOps.
• The server update process is an issue. You have to install a new server when there is an update. Then you decide to migrate or not. Migration is a scary word to me. I would like to see install over options, but I would like to also see auto-updates or a manual check for updates with an install option if there is one. Even if just for new checkers, like spam software does.

Pros to Highlight

• Team focus (read below).
• The built-in real-time refactoring for C++ is great. I need that little red arrow to tell me what is going on.
• The ability to relate issues across all files (traceback) and references is fantastic. VS will help you out with simple stuff line-by-line but the tracer goes across the broader issues application wide.

  

• Documentation. It’s very nice. Especially in reference to the checker code details.
• Nice that you can see a quick view of the code in the web app, without opening Visual Studio.
• The reporting and analytics was very important and useful for me.
• The static code analysis in VS tells you the severity of issues. This helps draw focus and is in the web app as well.
• Pro/Con you can write your own checkers. Great for established corporate standards or team wide design standards. Downside is they do not seem very easy to do and change control around them is a bit complex.
• Security built into checkers is based on standards CCSLP. So it is relatively static.  I wanted more and then I found out there was an integration with Maven. This is very cool and the component security I was looking for. Although I wish this was default.
• The Cyclomatic complexity metric is great. It helps teams focus on particular issues and to monitor the relationship between issue complexity and releases over time.
• They have integration with ticketing systems like Bugzilla, Jira, Rational Team Concert via a hook system. You will also have reference URLs bi-directional between the two systems. From my point of view this would be mandatory, as you do not want to have potential ticketing in two different places.

Wish List

• Public Custom Checkers Library. Like a listing of people who have written their own checkers for others to benefit from. Or general ones written by the Klocwork professional services team.
• I kind of wish there was an IIS version so I could more easily get it to work with my .NET web applications. But really the proper setup would be to run your application on its own server and Klocwork on its own, where analysis of code is sent from one to the other.
• Error monitoring. Being able to take data from entire stack traces would be cool. This would be done by automatically inserting a call in the exception blocks to the web app.
• Auto-update of checkers like you would get in anti-virus software.
• PHP support.
• Pre-Configured AWS or Azure instances.

Teams

One of the biggest things for me was not an individual feature but the broader concept of team-wide analytics. I preach in the world of DevOps the need to be holistic and look not only at individual developer code, but the entire pipeline.

Usually code analysis is very individualistic, but Klocwork allows you to abstract from the individual to the team. You can look at release trending, overall code quality and relationship to backlog to errors for example.

Build by build you collect analytics and with some work you can integrate it into ALM or Continuous Integration (CI) processes. You would do this by adding analysis scripts to your compile and run process or by adding this to your TFS (inject), Jenkins, Octopus Deploy, Go, etc. build scripts.

The analytics, oh the analytics. They tell a great story. For example look at the below dashboard I built.

As a team lead this chart tells me a lot of things. First off, our error types (top right) are relatively evenly distributed. I would want to know the most common errors so we could spend some time in a stand-up meetings focusing on those. I was embarrassed to find out that in both of my applications 60% of the issue distribution was empty exception blocks (CS.EMPTY.CATCH: 11).

Top left is the most interesting chart. Each bar represents a build and the number and severity of issues in that build. If you look at the last two releases we can see there is a big jump both in volume and severity, which could be a clear indication that the team is being pushed too hard! Or a new feature has some big issues that are persistent.

This is such powerful stuff to get out of the weeds and focus on team and code quality overall.

Wrap Up

I was pretty apprehensive going in, but I must admit that I really enjoyed working with Rogue Wave’s Klocwork product. (However, I do think there is a need for improved UX and modernization of their own delivery and application features.)

Part of the reason I did not become a mainstream coder was because I was too lazy to refactor my code. Thus my code was, well ... not great.

Klocwork is kind of like my code spell check, or professional editor. It just makes me sound good. And if I were part of a larger dev team it would be quite a powerful management tool.