Workspace Template Manager to Intelligently Update Workspace Information of a Build

• Download source code - 898.44 KB

Introduction

In Team Build, we target to cover the complete code base of our product with continuous builds, i.e., make an attempt for continuous integration. In such cases, builds are triggered as soon as check-in happens and the stake-holders are informed about the build failure, success and other useful information. That’s really cool. Let us take a step towards the real world. It’s not useful or practical to integrate the complete code base within one build, simply because it may take several hours to build the complete code and will give the information only at a higher level, e.g. - The build breaks and the reason could be any one of the several 50 associated change-sets which happened since the last build. This is why we tend to create more number of continuous builds and try to club small solutions together so that an average build takes 2 minutes to 20 minutes and we get the information at a lower level directly pointing out the developer and the change-set that broke the build. So, understanding the fact that we need several smaller builds to cover the complete code base by logically grouping them, we must agree that at the same time we need to have a build which integrates several continuous builds and executes them together as a group so that we can build some solutions together and check for integration errors and many more (probably on a single dedicated build agent)!!! Let’s call the integrations build as hourly build, assuming it will try to integrate a couple of smaller continuous builds every hour.

Having said that, I can write my continuous build definitions code in the .targets file, instead of writing the MSBuild script in the TFSBuild.proj, so that it can be reused in the hourly build, but here comes the problem of workspace mappings. In order for the hourly build to work, it needs all the workspace mappings of all the continuous builds it’s going to execute and believe me, it’s no point adding them manually and maintaining them. The workspace mappings of the continuous builds itself keep on changing very rapidly and there will be lots of conflicts while considering the mappings of all the continuous builds together in a single build. So we need some way to get this workspace mapping automatically added in the hourly builds as soon as they are executed. This article talks about implementing a Workspace Template Manager and plugging it in the correct event of the build cycle, such that it performs the expected task and team build accepts the changes done in the same build definition which it has already started processing!

Background

I faced this challenge in the middle month of the third quarter of 2009. At first, it looked to me as a solvable problem, but I could not make it work. I tried all approaches like using the vast API of TFS, MSBuild scripts, tf.exe command like utility and Googling of course. Then my development lead came in and we tried together for some days. Afterwards, fortunately, again I took some more time and with good luck, this is the solution I implemented and it worked without any complaints. All those working on continuous / hourly kind of framework for their builds must have faced this problem. There is no solution present yet! So, here you go for the solution.

Visualize the Challenge

The above image on the left, shows a typical product source control structure, having lots of code like multiple solutions, projects for Business Logic and Data Access Layer Code, etc. On the right, it shows the continuous builds targeting these solutions separately. This is what we just discussed above, that there are several continuous builds for covering the smaller parts of source code independently. In the middle is shown the workspace mappings of one of the continuous builds. Similarly, we have a different workspace template for every build. Here, we need to have one hourly build definition 'Hourly.All_BL_With_DAL' which builds all the solutions of Business Logic and DAL and hence needs to have all the workspace mappings present in all of these builds, as shown below, regardless of the frequency other builds are changed and the expected conflicts across build definitions.

Implementing the Solution

These workspace mappings are a difficult challenge to play with. As soon as a build is queued, among the starting steps team build internally performs lot of tasks like assigning workspace name, its initialization, mappings, etc. So manipulating the workspace gives lots of run time errors and in cases even no error and no results as well. These mappings can be played in only one event in the build cycle and only with the below discussed API, it's then that the changes are accepted by the team build and processed immediately within the current build. The below approach will process the workspace mappings in the build as soon as it starts getting processed, i.e., it will execute the hourly build with the latest workspace mappings required and that is what we wanted.

We will implement the solution by writing our logic in a custom task to be called by the MSBuild script in the correct event ‘BeforeInitializeWorkspace’ of the build cycle.

The below custom task – WorkspaceTemplateManager, can be built and placed as a .dll in the TeamBuild folder of the build agent, so that it can be referred from the MSBuild scripts. The code is very well explained with the inline comments. This task has an overridden Execute method which performs these steps:

1. Find out all the continuous builds script files referred from this hourly build. 
2. Get the build definitions object model instance using tfs API. 
3. Initiate the process of getting the workspace details from all the continuous builds and updating them in the hourly builds and take care of all conflicts. This can be better read from the below code inline / block comments.

using System;
using System.IO;
using System.Xml;
using Microsoft.Build.Utilities;
using Microsoft.Build.Framework;
using System.Collections.Generic;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.VersionControl.Client;

namespace RA.TeamFoundation.Build.Tasks
{
    public class WorkspaceTemplateManager : Task
    {
        // The team foundation server to run tasks against.
        public string TeamFoundationServerUrl { get; set; }

        // The name of the team project to get build definitions from.        
        public string TeamProject { get; set; }

        // Current Build definition name, to be able to access its 
        // run time object using API
        public string BuildDefinitionName { get; set; }

        // Current Build proj file i.e. a .targets file or a 
        // .proj file to read the msbuild scripts at run time
        public string BuildProjectFilePath { get; set; }

        // A bool to update the msbuild scripts whether this task successfully executed 
        // or otherwise, if a build break is required.
        private bool _sucessfullyExecuted;

        public override bool Execute()
        {
            try
            {
                // get the list of all .targets files imported 
                // in the current build definition script. 
                // This will be used to find out all the continuous builds 
                // attached to this hourly build
                List<string> continuousTargetsFiles = GetAllContinuousTargetsFiles();

                // get the list of all other build definitions 
                // attached to the current build
                List<IBuildDefinition> contBuildDefinitions = 
				RetriveBuildDefinitions(continuousTargetsFiles);

                // Initiate the process of identifying the workspace mappings "correctly" 
                // from all the continuous builds and update them 
                // in the current hourly build
                IdentifyAndUpdateWorkspaceTemplate(contBuildDefinitions);
                _sucessfullyExecuted = true;
            }
            catch
            {
                // Any exception will actually fail the task
                _sucessfullyExecuted = false;
            }

            return _sucessfullyExecuted;
        }

        //
        // This function iterates through all the continuous build 
        // definition's workspace mappings
        // and finds out if this mapping can be directly added / updated
        // or there are conflicts with other mapping types and resolves them
        // It also sees the major possibility of skipping the edit part, 
        // as it's already present
        //
        private void IdentifyAndUpdateWorkspaceTemplate(List<IBuildDefinition> 
		selectedContinuousBuildDefinitions)
        {
            // get the current hourly build and its workspace template
            IBuildDefinition hourlyBuildDefintion = GetBuildDefinitionByName
		(TeamFoundationServerUrl, TeamProject, BuildDefinitionName);
            hourlyBuildDefintion.Refresh();
            IWorkspaceTemplate currnetWorkspaceTemplate = hourlyBuildDefintion.Workspace;

            // iterate through all the continuous build definition to see 
            // the possibility of their mappings to add / edit in the hourly build
            foreach (IBuildDefinition selectedContBuildDefinition 
			in selectedContinuousBuildDefinitions)
            {
                // iterate through each and every mapping of the continuous build
                foreach (IWorkspaceMapping continuousMappingToAdd 
			in selectedContBuildDefinition.Workspace.Mappings)
                {
                    // assume it's safe to directly add this mapping 
                    // to the parent hourly build
                    bool blnSafeToAddNewMapping = true;

                    // iterate through all the existing mappings of the hourly workspace 
                    // to find out any possibility of conflict and 
                    // remove / skip / edit that mapping accordingly
                    foreach (IWorkspaceMapping currentMapping in 
				currnetWorkspaceTemplate.Mappings)
                    {
                        // if the mapping to be added is already present, 
                        // then we need to resolve the conflict
                        if (currentMapping.ServerItem.Equals
				(continuousMappingToAdd.ServerItem))
                        {
                            // There are 4 combinations possible here 
                            // for conflicts between already added mappings
                            // Case No: Existing Mapping Type -> 
                            // New Mapping Type => Action to be taken
                            // Case 1:  Cloak - Cloak => No Action required
                            // Case 2:  Cloak - Map   => 
                            // Remove the existing mapping and read the new mapping
                            // Case 3:  Map   - Map   => Consider the local items. 
                            // If local items are not same, 
                            // remove the complete mapping and read it again, 
                            // else no action required.
                            // Case 4:  Map   - Cloak => No action required. 
                            // This is to cover all cases for conflicts between 
                            // multiple build definitions
                            if ((currentMapping.MappingType == 
					WorkspaceMappingType.Cloak) 
                			&& (continuousMappingToAdd.MappingType == 
					WorkspaceMappingType.Map))
                            {
                                // Case 2:
                                currnetWorkspaceTemplate.RemoveMapping(currentMapping);
                                blnSafeToAddNewMapping = true;
                            }
                            else
                            {
                                // Case 3:
                                if (!String.IsNullOrEmpty(currentMapping.LocalItem)
                                    && !String.IsNullOrEmpty
					(continuousMappingToAdd.LocalItem)
                                    && !currentMapping.LocalItem.Equals
					(continuousMappingToAdd.LocalItem))
                                {
                                    currnetWorkspaceTemplate.RemoveMapping
							(currentMapping);
                                    blnSafeToAddNewMapping = true;
                                }
                     else
                                {
                               // Case 1: and Case 4:
                                   blnSafeToAddNewMapping = false;
                                }              
                            }

                            // safely break the loop, as the server 
                            // item string is always unique in the workspace. 
                            // (This conforms the workspace concepts as well)
                            break;
                        }
                    }

                    // Reading the mapping if was not present or for Case 2 or 
                    // for Case 3
                    // add the new safe workspace mapping to the hourly build, 
                    // with the same local item, mapping type and depth as well
                    if (blnSafeToAddNewMapping)
                        currnetWorkspaceTemplate.AddMapping
			(continuousMappingToAdd.ServerItem, 
				continuousMappingToAdd.LocalItem, 
                            continuousMappingToAdd.MappingType, 
				continuousMappingToAdd.Depth);
                }
            }

            // save the hourly build definition
            hourlyBuildDefintion.Save();
            hourlyBuildDefintion.Refresh();
        }

        //
        // Gets a list of build definition names from the .targets files. 
        //
        private List<IBuildDefinition> RetriveBuildDefinitions(List<string> targetsFiles)
        {
            // connect to the tfs server, build server and version control server
            TeamFoundationServer tfs = new TeamFoundationServer(TeamFoundationServerUrl);
            IBuildServer buildServer = 
		tfs.GetService(typeof(IBuildServer)) as IBuildServer;
            VersionControlServer vcs = 
		tfs.GetService(typeof(VersionControlServer)) as VersionControlServer;

            // query all the build definitions of the team project
            IBuildDefinition[] allBuildDefinitions = 
		buildServer.QueryBuildDefinitions(TeamProject);

            // select list of continuous build definitions we are interested in
            List<IBuildDefinition> selectedBuildDefinitions = 
					new List<IBuildDefinition>();

            // iterate through all the target files and all the 
            // build definitions and find out the 
            // builds configured with these target files.
            foreach (string targetsFile in targetsFiles)
            {
                foreach (IBuildDefinition buildDefinition in allBuildDefinitions)
                {
                    string path = buildDefinition.ConfigurationFolderPath + 
				"/" + targetsFile;

                    if (!vcs.ServerItemExists(path, ItemType.File))
                        continue;

                    selectedBuildDefinitions.Add(buildDefinition);
                }
            }

            return selectedBuildDefinitions;
        }

        //
        // This function returns the object model 
        // build definitions instance from its name
        //
        private static IBuildDefinition GetBuildDefinitionByName
		(string teamFoundationServerUrl, string teamProjectName, 
		string buildDefinitionName)
        {
            // connect to tfs and build server
            TeamFoundationServer tfs = new TeamFoundationServer(teamFoundationServerUrl);
            IBuildServer buildServer = tfs.GetService
			(typeof(IBuildServer)) as IBuildServer;

            // retrieve an instance of the build definition from its name
            IBuildDefinition buildDefinition = 
		buildServer.GetBuildDefinition(teamProjectName, buildDefinitionName);
            return buildDefinition;
        }

        //
        // This function returns the list of all proj / .target files 
        // imported by the current build definition
        // i.e. the list of all other continuous builds which will be 
        // executed as part of this build.
        //
        private List<string> GetAllContinuousTargetsFiles()
        {
            // get the list of imported project nodes
            XmlNodeList importedProjects = GetImportedProjects();

            List<string> targetsFiles = new List<string>();
            foreach (XmlNode importedProject in importedProjects)
            {
                // get the list of imported project file names
                string projFileName = GetProjectFileName(importedProject);
                targetsFiles.Add(projFileName);
            }

            return targetsFiles;
        }

        //
        // This function returns the list of all projects to be 
        // executed by the current build
        //
        private XmlNodeList GetImportedProjects()
        {
            // Load the current build script XML file to process its XML nodes.
            XmlDocument projectFile = new XmlDocument();
            projectFile.Load(BuildProjectFilePath);

            XmlNamespaceManager nsmngr = new XmlNamespaceManager(projectFile.NameTable);
            nsmngr.AddNamespace("pr", 
		"http://schemas.microsoft.com/developer/msbuild/2003");

            // get the project/import nodes 
            // i.e. all the .proj or .targets files imported.
            XmlNodeList importNodes = projectFile.SelectNodes
				("pr:Project/pr:Import", nsmngr);
            return importNodes;
        }

        //
        // This function returns the imported project file name
        //
        private static string GetProjectFileName(XmlNode importedProject)
        {
            XmlAttribute projectAttriubute = importedProject.Attributes["Project"];
            string targetsFile = Path.GetFileName(projectAttriubute.Value);
            return targetsFile;
        }
    }
}

Then we need to invoke this custom task from the hourly build. This is done by invoking it from the BeforeInitializeWorkspace target and passing the required parameters. Below is the sample hourly build targets file to show this invocation of task and inclusion of continuous builds targets file in it. This also has been decorated with inline comments.

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="DesktopBuild" 
    xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">

  <!-- The location where all the .targets files of the product are kept-->
  <PropertyGroup>
    <TargetsLocalRoot>$(MSBuildExtensionsPath)\Microsoft\VisualStudio\TeamBuild\Main
    </TargetsLocalRoot>
   </PropertyGroup>


 <!-- Declare the custom task name and the DLL name which will be 
	called from the build events / targets -->
  <UsingTask 
 TaskName="RA.TeamFoundation.Build.Tasks.WorkspaceTemplateManager"
 AssemblyFile="$(TargetsLocalRoot)\RA.TeamFoundation.Build.Tasks.dll"/>

  <!-- This is the only event in the build cycle where the workspace 
	template manager task can be called and executed 
    	and changes will be accepted and immediately processed by the 
	team build within the currently executed build -->
  <Target Name="BeforeInitializeWorkspace">
    <WorkspaceTemplateManager
      TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
      TeamProject="$(TeamProject)"
      BuildDefinitionName="$(BuildDefinition)"
      BuildProjectFilePath="$(TargetsLocalRoot)\Hourly.All_BL_With_DAL.targets" />
  </Target>
  
  <!-- The below shows that this hourly build definition actually will 
	process all the below 4 continuous builds -->
  <Import Project="$(TargetsLocalRoot)\Continuous.BL_Controllers.targets" />
  <Import Project="$(TargetsLocalRoot)\Continuous.BL_CoreLogic.targets" />
  <Import Project="$(TargetsLocalRoot)\Continuous.BL_WebServices.targets" />
  <Import Project="$(TargetsLocalRoot)\Continuous.DAL_DataAccessCode.targets" />
 
</Project>

Conclusion

So, using the above solution will actually give the latest workspace mappings to the hourly builds, every time before they are processed, and will take care of any latest change in the continuous builds and will reflect it immediately in the hourly build.