QuickBuild .NET Solutions

Abstract

In complex applications, we often end up having huge number of projects to be built before we proceed with running/debugging/ testing an application locally. The process of building all the projects locally is often a manual process, where a developer takes the latest version of code, builds each project/solution in a particular order, making sure not breaking the project references. This process is usually time consuming and error prone as several manual steps are involved.

This paper provides the approach used for creating the local build for complex applications having huge number of projects/solutions. The approach presented makes the development/debugging/testing process more efficient, highly accurate and less time consuming. If an application having large number of projects/solutions and the bug fixing and delivery is much frequent, then this approach will be the best way to ensure that the fix delivered is accurate and compatible with other modules. The approach used in this paper, building the whole application locally will be a single click process and the build files will be located in a single folder from where the application runs.

We are currently using the proposed approach in our project with very successful results reducing the initial set up issues and intermittent builds to a great amount of time. This approach is incorporated all over the team of developers there by saving a significant amount of time all over the project. We implemented this solution using Microsoft Build tool available with Microsoft .net Framework 3.5, Microsoft Visual Studio 2008 development environment and Team Foundation Server 2005 as configuration and source management tool.

Introduction

When there are large numbers of projects involved in an application, often, setting up the environment for bug fixing/debugging/testing involves building of the application locally. Process of building the application locally would be as follows:

• Developer gets the latest source code to the local machine in a work space.

• Build projects/solutions locally.

• Identify the projects/solutions locations.

• Open the project/solution in Visual Studio and build the same.

• Copy the build output (.dll/.exe) to a common location.

• Repeat the same process for all the projects/solutions in the application making sure not to miss the order.

• After getting the application ready to be debugged locally, then the developer can proceed with fixing the bug.

• Once the bug is fixed the developer needs to build the whole application, following the steps above, making sure that his fix is not causing any compatibility issues with other modules.

• Once the build is successful, he checks in the code for further process of deployment and release.

• The same process needs to be followed by all the developers in the team.

As we can see, the above process involves manual steps which if not done correctly will result in non-compatible and erroneous code. Finally during the release process, the build may fail or result in application failure due to non-functioning of the requirement. Below mentioned issues are quite common during the check in process of the code by the developer.

• After getting the latest version, the developer needs to identify the location of the projects/solutions and build them one by one.

• While building, even if a single project is not built, then there is a risk of not able to complete the whole build, there by resulting is redoing the build process.

• If the developer checks in the fix without building locally, then it may result in the server build failure, used for deployment. This raises concern like, whose fix broke the code, and what it takes to fix the same.

• If the application contains a large number of projects, the building them locally is a time consuming process, and the same needs to be followed by the whole team resulting in cumulative loss of time.

This paper attempts to address the above mentioned issues by providing an approach to build all the projects/solutions locally with a single click, here by referred to as QuickBuild. The approach to build locally using the MSBuild platform under .net Framework discussed in this paper provides a highly flexible and fast mode of build process involving multiple projects. This approach was tested and implemented. The new process helped save time in developing and ensuring the right fix.

Need for QuickBuild

Need for developing this quick build approach originated due to the requirement of the very frequent bug fixes and enhancements as we are following iterative delivery model. Our project contains around 360 projects distributed in 19 solutions. A developer producing a fix or enhancement needs to build the entire application to ensure that the code he checks in is compatible with the other modules, as change in one project, results in change in other dependent projects and thus a typical fix would also involve in building the entire application. Considering a project of such a large size will contain about a dozen developers. Each developer facing the same build issue on their local machines, and every one taking such a long time, directed us to create quick build. We have analyzed few points before switching to the quick build process, for producing entire application build output, on local machine and the steps involved in both the process.

To summarize, below are the major drivers behind this approach:

1. Frequent releases happening due to iterative model.

2. Complex project with many projects and solutions.

3. Time consuming process for every developer.

4. Manually building project by project is error prone.

5. Setup once by TFS admin, build forever by developers.

Details of Quick build setup using MSBuild

1. Implementing quick build process

Our project is built on .net Framework 3.0 with C# as the programing language and TFS as configuration management.

Any of the above can be modified, like; this process can be used in any of the .net platform with any .net programming language, and any other configuration management tool. When a developer checks in his code, he needs to make sure that his code doesn’t break anything else in the system. To ensure this he has to build the entire application on his local machine before checking in the code. We have also incorporated a rule within the team that no one is going to check in without the build log. Collecting the build log of 360 projects and pasting the same in the check in comments is cumber some. Instead only one consolidated build statement, stating build successful would be an efficient way.

Below sub sections talk in detail about the steps needed to configure the quick build process and would be applicable to all .net versions, languages and version control systems.

2. Pre requisites

Before we begin the quick build process, we need to make sure if we have the following pre-requisites.

• .NET Framework is installed

• All the solutions/projects are loaded into the local machine

• The project output set to a common folder.

• Create a build file containing the list of solutions in a specific order.

Preparing the common folder

Since we have multiple projects, and as a standard the build output is stored in /bin/debug folder by default. Building each project and copying the output to a common folder for references will be a time taking as well risky process. Instead, change the output path for each project to a common folder like “C:\builds\”. With this, every project being build copies the assemblies to the output folder specified. There by other projects referencing these assemblies will need to be referred from this location. The assemblies need not be placed in a common location if there is only one solution entirely in the application, where as if there are multiple solutions, as in our case, then the output needs to be in a common location, for projects in other solutions refer these.

Why a Common Folder?

By default the output path will be bin\Debug\. It means that once the project is successfully built, the assemblies will be copied to this folder. Each project will have its own bin\Debug\ folder. When each project is built, all the assemblies will be loaded into the bin\Debug\ folders of each project. Accumulating them for setup/deployment project or to run the application would be tough. Specifying a common folder will allow the assemblies to be copied into the same folder there by eliminating the task of finding the folder and extracting the assemblies.

Prepare Solutions for the Projects per Module

It is easy to build projects related to a specific module. We need not worry about the build order with the projects in a solution. Solution takes care of the build order. Place all the related projects in a solution. At the same time we cannot place all projects in a single solution, reason being the machine will not be able to handle so many projects opened in a single solution. I can load all 360 projects in a single solution and use the standard build command available in Visual Studio. As a matter of fact there is no limit on number of projects in a solution. One of my solutions has 55 projects in it. But the standard approach is not more than 10-15. The less number of projects, the shorter the build time.

Make sure that all the projects are in one or the other solution. List down the build order for the solutions.

Prepare the build XML

Build file is an xml file with .build extension. The build file specifically contains the list of solutions to be built, source and destination folder locations and any additional tasks if any 

Defining the source folder

The below sample XML states how the source folder needs to be specified. “..\” mentions that the source folder is one level above the current location.

SourceFolder is the variable used all over the XML.

<PropertyGroup>
<SourceFolder>..\</SourceFolder>

Specifying targets

Targets to the build are the solution files. We need to specify the solution files in the below format.

<Target Name="Build">
  <MSBuild Projects="$(SourceFolder)\Creehan3\Framework\Framework.sln"  StopOnFirstFailure="true"   >
     <Output TaskParameter="TargetOutputs" ItemName="BuildOutput" />
   </MSBuild>
</Target>

Projects attribute in MSBuild tag specifies where the solution is available. Observe how the variable $(SourceFolder) is used.

The StopOnFirstFailure attribute specifies that if there is any failure then stop the build. Similarly specify all the solutions in the Target tag in the order you need to build them. On the whole the build file looks as below:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >
<PropertyGroup>
<SourceFolder>..\</SourceFolder>
</PropertyGroup>
<Target Name="Build">
<MSBuild Projects="$(SourceFolder)\solution1.sln"  StopOnFirstFailure="true"   />
<MSBuild Projects="$(SourceFolder)\solution2.sln" StopOnFirstFailure="true" />
<MSBuild Projects="$(SourceFolder)\solution3.sln" StopOnFirstFailure="true"  />
<MSBuild Projects="$(SourceFolder)\solution4.sln" StopOnFirstFailure="true"  />
</Target>
</Project>

This is a very basic script. It only builds your solution. But you can easily enhance it to add other important tasks, such as, running unit tests, packaging output, code coverage etc.

Running Quick Build

Quick build can be run from the visual studio command prompt with the following command:

Msbuild /t:Build /l:FileLogger,Microsoft.Build.Engine; logfile=build.log application.build

• /t specifies that the Build action has to be performed on the targets.
• /l specifies to log events from MSBuild.
• Logfile specifies the filename for the log.
• application.build specifies the build file (created in section 3.6)

Creating one click approach

For this we will be creating a batch file which triggers the MSBuild command.

Apart from invoking the MSBuild command we can perform other actions like copying pre-requisite assemblies to the common folder, writing the build output to a log etc.,

del C:\builds\*.* /Q > dependency.log
@echo Cleaning N: drive Complete

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\msbuild.exe /t:Build 
   /l:FileLogger,Microsoft.Build.Engine;logfile=build.log application.build 
@echo Build complete. 
pause

This batch file does the following tasks.

• Deleting all the files from the common folder, enabling us to have a fresh build every time.

• Set the working directory to visual studio command prompt.

• Run the MSBuild command.

• Write the log to build.log file

The build file and the batch files are created in such a way that they can be executed from the application root folder.

Placing this batch file in the application root folder and checking into TFS enables the developers to build on their local machine.

The above figure shows my project structure with QuickBuild folder containing a build file and a batch file.

After the build

Once the build is successful we can find the output in the common folder.

If the build is a failure we can find the log in the same folder where the QuickBuild batch file is available. The log file contains the complete build output for each project in the solutions. Using this we can identify where/why the build failed.

What a developer needs to do?

Set up an N:\ drive (common folder) on his local machine, a onetime task. Get the latest to a work space on his local machine. Locate the quick build folder and double click on the batch file. The build gets triggered and the build output placed in a common folder specified.

Further steps

QuickBuild is one of the fastest ways to get the build output considering references to projects with in the same application. This can be enhanced in many ways.

• We can modify the build file to include the creation of setup/deployment MSI’s.

• We can also write scripts to deploy the application to the location specified.

• Can integrate the quick build as one of the Visual Studio menu items so that it can be run from visual studio.

• Add or remove one or more solutions as the project demands.

Challenges Faced

Common folder for every developer

Every developer should be having the same common folder and at the same location. This would not be possible for all the developers. Since the output folder for every project is hardcoded to the common folder, and if that common folder does not exist on the developer’s machine, the build fails.

To eliminate this challenge, we have created a folder and mapped it to N:\ drive on the local machine. Every project will be pointing the output to the N:\ drive. Every developer will be creating any folder at any location, but mapping it to N:\ drive solves the problem.

Data Sample

The Table 2 shows the different type of projects our application has currently.

Table 3 below compares time taken in best and worst cases for manual and quick build.

Considering a typical scenario where a new developer needs to fix a bug and check in the code into TFS, making sure that nothing else breaks in the system, the following approach needs to be taken:

1. Get the latest from TFS.

2. Perform local build to run/debug the application locally

3. Fix the code

4. Perform local build to ensure the fix doesn’t break any other module.

5. Check in the code.

To perform the above steps in a best case scenario, a developer would take about 8-10 hrs.

Whereas using the approach specified, he would not take more than 30 min.

Considering a team of 10 developers, performing the same scenario specified above, the total man-hours would be 100.

Whereas following the approach specified it would be 5 hrs. thereby gaining a total effort to more than 11 man days.

In case of quick build the best case scenario is where no manual error as mentioned in “Section 4 – Challenges Faced” happens. Worst case scenario is when some manual error occurs like not getting the latest version, not mapping the common output folder etc., In case of manual build; worst case scenario is more severe since error can happen while missing the sequence of build. In this case, it is often hard to debug why an error has happened and whole cycle sometimes gets repeated.

The above mentioned sample can change also depending on the size and the number of components but the time of Automated Build will nearly to be same. So bringing this practice in our delivery made the local build very fast as compared to the general practice we adopted earlier. This not only reduces the build time for the developer but also every time whenever he has a code change.

Advantages of Quick Build

Below are the main advantages we are getting by implementing this approach

• No need to worry about the build sequence as the build sequence will be predefined.
• No need to load multiple solutions/projects in visual studio.
• No need to open visual studio to perform this build.
• Run the application directly from the common root folder set for each project.
• A single command would build all the projects/solutions.
• A onetime creation of the configuration file can be used across the machines and developers in the team.

Achievement

The current quick build has been implemented to address a long pending pain faced by the developer, in turn the customer, for the delay in the delivery.

The innovative approach to build locally with one click has resulted in huge time saving during the development process. Additionally it has made the entire process reliable, robust and lesser error prone.

Because of the above advantages, approach presented in the paper has been distributed to different teams and the feedback received has been very encouraging.

Acknowledgements

My sincere thanks to Creehan and company, who provided an insight into this type of build, over which I have enhanced it. I also thank the company for providing the required platform to share my ideas.

References

1. MSDN.
2. “Professional.Team Foundation Server” by Wrox publications.
3. “Pro Visual Studio Team System Application Lifecycle Management” by Apress publications.
4. http://msbuildextensionpack.codeplex.com/