Gated Check-in in Git Repository

Introduction

The idea is to implement the Gated Check-in (Visual Studio TFS) in Github's way. The code should be integrated only when the unit tests and code quality parameters are met.

This is to explain how we can query the Sonarqube server for a given project and check if the Sonarqube quality gates are passed for the project. Depending on this result, we can fail the pipeline manually. Also, fail the pipeline if UTs for the project fail. This will mandate the developer to fix the quality issues or UTs prior to merging the code to master branch.

Background

Failing the pipeline on UT fail was simple enough. But getting the status from Sonarqube was quite challenging. I came across this article to do the same but it was for the Linux environment.

Using the Code

Every project will have the .yml file at the root location and the git runner will look for this file to perform the pipeline related activities. We will be focusing only on those changes needed in the yml file for achieving gated check in.

In TFS, the code is not checked in until all UTs are passed. But in Git, the pipeline is triggered only after the changes are checked in. We can make use of Merge Request to merge the branch into master branch. During this, if the latest pipeline of the feature branch fails, the merge request can't be completed until the pipeline is green. So we make the pipeline fail if the UTs and code quality parameters are not as expected. As a process, it should be mandated to make use of creating merge request feature for any sort of merging.

The following are the code blocks to make the pipeline fail when:

1. UTs Fail

'&"<OpenCover Exe Location>" "-output:<Output file location>" 
-register "-target:<NUnit Console Exe Location>" 
"-targetargs:<Dlls to be analyzed with spaces if multiple> 
/result=<NUnit result file location>;format=nunit2" -returntargetcode'

The above code uses Open cover to check the coverage and NUnit uses to run the UTs. "-returntargetcode" returns the status of the UTs execution as Pass or Fail. If any of the UTs fail, the pipeline will fail automatically.

2. Sonarqube Quality Parameter Fails

Sonarqube can be configured with some quality parameters after logging in as admin. By default, some parameters will be set and these can be altered as per the project requirements.

The yaml code for querying and receiving the status from sonarqube is as below:

- $url = (findstr "ceTaskUrl" "<report-task.txt location>").Substring(10)
- sleep 10 #Need some buffer time to get the report updated from sonarqube analyzer
- $response = &"<Curl exe location>" -u <SonarAdminUserName>:<Password>
  $url #using curl to login to sonarqube to check analysis ran properly or not.
  Using sonar admin credentials/token
- $sonardata = $response | ConvertFrom-Json #converting returned data to json
- $sonarBuildStatus=$sonardata.task.status
- |
      if ("$sonarBuildStatus" -eq "SUCCESS"){
          echo "SONARQUBE ANALYSIS IS SUCCESSFUL"
          $sonarAnalysisId= $sonardata.task.analysisId
          $projurl = (findstr "serverUrl" "<report-task.txt location>").Substring(10)
          $projNewUrl = $projurl+"/api/qualitygates/project_status?
                         analysisId="+$sonarAnalysisId
          $projresponse = &"<Curl exe location>" -u <SonarAdminUserName>:<Password>
                          $projNewUrl
          $sonarprojdata = $projresponse | ConvertFrom-Json
          $sonarProjStatus=$sonarprojdata.projectStatus.status
          if ("$sonarProjStatus" -eq "ERROR"){ #Checks if the project
                       has meet all the quality gates specified
              echo  "SONARQUBE QUALITY GATES FAILED FOR $CI_PROJECT_NAME"
              echo $sonarprojdata.projectStatus.conditions
              exit 1 #breaks the build for violations
          }
          else{
              echo "SONARQUBE QUALITY GATES SUCCESSFUL FOR $CI_PROJECT_NAME"
              echo $sonarprojdata.projectStatus.conditions
              exit 0
          }

      }
      else{
          echo "SONARQUBE ANALYSIS FAILED"
          exit 1 #breaks the build for violations
      }

Let's understand how this works:

1. SonarBuildRunner runs on a given project, considering all the exclusions it runs based on its profile set.

2. Once the above process gets over, it creates a hidden folder in the start up project of the application being analysed, with a file name "report-task.txt".
   Example: <project location in git runner>\.sonarqube\out\.sonar\report-task.txt

   Sample report-task.txt looks like:

   projectKey=<project key>
   serverUrl=http://localhost:9000
   serverVersion=7.7.0.23042
   dashboardUrl=http://localhost:9000/dashboard?id=<project key>.
   ceTaskId=<TaskId>
   ceTaskUrl=http://localhost:9000/api/ce/task?id=<TaskId>

   Also, Report-task.txt will tell only the analysis done by sonarqube on the project. It doesn't give any information about the project. It gives information about Sonarqube server URL, version, etc. as seen above.

3. The above report-task.txt contains a URL called "ceTaskURL". This URL has to be accessed to check if the Sonarqube was able to analyze the project or not. Code for the same is as below:

   - $url = (findstr "ceTaskUrl" "<report-task.txt location>").Substring(10)
   - sleep 10
   - $response = &"<Curl exe location>" -u <SonarAdminUserName>:<Password> $url
   - $sonardata = $response | ConvertFrom-Json

   The code first looks for "ceTaskUrl" value from the report-task.txt and sleeps for 10s as Sonarqube takes some time to analyze and update file. Once the file is updated, we access URL using curl. Also the URL can be accessed using sonar admin credentials or the admin token. The response from the URL is converted from Json to Powershell compatible values.

4. When this URL is accessed, it returns the status analysis of the project along with some other information about the analysis of the project. In the response, we can check if the project has passed all the quality gates set in the Sonarqube.
5. We check the status of the analysis by verifying the value of "sonarBuildStatus", If it is success, the analysis of the project was successful, otherwise the Sonarqube analysis wasn't able to perform the analysis on the project.

If the analysis was successful, we take the analysisId from the data returned from the URL and form a new URL as "http://localhost:9000//api/qualitygates/project_status?analysisId=<analysisId>" . This gives us the information about the project being analysed, based on the quality gates set it returns if the project passes the quality gates or fails to qualify. If the above result is ERROR, it means the project didn't pass quality gates and the git pipeline will be made to fail by using exit 1. Code is as below:

- $sonarBuildStatus=$sonardata.task.status
  - |
        if ("$sonarBuildStatus" -eq "SUCCESS"){ 
            echo "SONARQUBE ANALYSIS IS SUCCESSFUL"
            $sonarAnalysisId= $sonardata.task.analysisId
            $projurl = (findstr "serverUrl" "<report-task.txt location>").Substring(10)
            $projNewUrl = $projurl+"/api/qualitygates/project_status?
                          analysisId="+$sonarAnalysisId
            $projresponse = &"<Curl exe location>" -u $SONAR_ADMIN_TOKEN $projNewUrl
            $sonarprojdata = $projresponse | ConvertFrom-Json
            $sonarProjStatus=$sonarprojdata.projectStatus.status
            if ("$sonarProjStatus" -eq "ERROR"){ #Checks if the project 
                 has meet all the quality gates specified
                echo  "SONARQUBE QUALITY GATES FAILED FOR $CI_PROJECT_NAME"
                echo $sonarprojdata.projectStatus.conditions
                exit 1 #breaks the build for violations
            }
            else{
                echo "SONARQUBE QUALITY GATES SUCCESSFUL FOR $CI_PROJECT_NAME"
                echo $sonarprojdata.projectStatus.conditions
                exit 0
            }            
        }
        else{
            echo "SONARQUBE ANALYSIS FAILED"
            exit 1 #breaks the build for violations
        }

If the Sonarqube result is as Error, all the list of Sonarqube Quality gates are displayed in the pipeline console along with the corresponding values and as we are doing exit 1, it will fail the pipeline.

History

• 6^th November, 2019: Initial version