Introduction
Microsoft's hosted Team Foundation Server (TFS) service (tfs.visualstudio.com) does not support public repositories. A custom build activity can be used to mirror a TFS Git repository to public Git services such as GitHub (www.github.com) or Bitbucket (www.bitbucket.com). This seamlessly provides convenient public access to a codebase, while maintaining control over the release of the repository to the public.
Required Working Knowledge
- Team Foundation Server build processes and activities
- Git
- Visual Studio 2013
Table of Contents
- Problem
- Requirements
- Background
- Possible Approaches
- Solution
- Due Diligence and Critical Risks
- End Notes
Resources
Prototype Visual Studio Solution
1.0. Problem
Microsoft's hosted Team Foundation Server (TFS) service does not support public repositories.
2.0. Requirements
- As part of a build process mirror a TFS Git repository to a read-only public Git repository
- Support hosted Git services from GitHub and Bitbucket
3.0. Background
Team Foundation Service supports hosting and building Git repositories. The build service's standard Git Build Definition GitTemplate.xaml uses the build activity Microsoft.TeamFoundation.Build.Activities.Git.GitPull to clone a repository and checkout a branch. The checked out branch is then built.
Under the surface the service depends on LibGit2Sharp to interact with Git [1] (github.com/libgit2/libgit2sharp). "git.exe" is not installed on the build controller [2].
TFS replaces the standard LibGit2Sharp connection plumbing with a custom implementation that is able to authenticate against the service. TFS creates a custom LibGit2Sharp.SmartSubtrtansport which wraps around an HttpClient. The HttpClient in turn is passed a message handler to take care of authentication (Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler). See Figure 3-1.
Figure 3-1. Excerpt of "TfsSmartSubtransport".
private TfsSmartSubtransport.TfsSmartHttpClient BuildHttpClientForUri(Uri serviceUri)
{
VssCredentials vssCredential = VssCredentials.LoadCachedCredentials(
CredentialsStorageRegistryKeywords.Build, serviceUri, false, CredentialPromptType.DoNotPrompt);
VssHttpRequestSettings vssHttpRequestSetting = new VssHttpRequestSettings();
vssHttpRequestSetting.ExpectContinue = false;
TfsSmartSubtransport.TfsSmartHttpClient tfsSmartHttpClient =
new TfsSmartSubtransport.TfsSmartHttpClient(
new VssHttpMessageHandler(vssCredential, vssHttpRequestSetting));
tfsSmartHttpClient.DefaultRequestHeaders.Add("User-Agent",
string.Concat("git/1.0 (Microsoft Git Client [Team Build] ",
TfsSmartSubtransport.s_assemblyVersion, ")"));
tfsSmartHttpClient.Timeout = TimeSpan.FromMinutes(30);
return tfsSmartHttpClient;
}
Although Team Foundation Service supports Git repositories there are limitations. The Git build definition cannot be changed through Visual Studio 2012. Visual Studio 2013 is currently required to select a different build template.
The service supports having multiple Git repositories in a team project. But there appear to be issues when repositories have the same name as another repository or another team project. This becomes a problem when selecting a repository and branch in the build definition. It also is a problem when programmatically working with TFS, where the incorrect repository
URL is returned.
There also is limited client support for more than one repository in a project. Both Visual Studio 2012 and the the online web portal only display the primary repository.
4.0. Possible Approaches
On the surface it appears that we should be able to create a custom build activity that uses LibGit2Sharp to mirror a TFS
repository to another service. Unfortunately we quickly run into issues.
The Git Build Definition does not contain the same set of build activities or variables that are part of a traditional Team Foundation Version Control (TFVC) build process. There is no "workspace" and there is little to no documentation on writing a custom build activity that interacts with the Git repository and build service using the TFS API.
Moreover, the LibGit2Sharp library is a work in progress and is incomplete. The library does not have all of the functionality we would require. And even if we could use the LibGit2Sharp library in its current form, much of TFS' custom connection plumbing is internal or private. We would need to depend on reflection against a code base that is changing, which is asking for trouble.
An alternative approach is to use the "git.exe" shell in place of LibGit2Sharp. Although "git.exe" is not installed on the build controller, there is nothing preventing us from uploading the executable and dependent files to TFS. The issue then becomes authentication. The only way to make this work would be to enable "alternative credentials" in TFS and set the username and password as values in the build definition. In this case it would be simpler to just use Git outside of TFS to copy the repository.
Another approach is to use a different library in place of LibGit2Sharp and replace the connection plumbing so that the library can work with TFS. The only real alternative .Net library is NGit (github.com/mono/ngit), which is a port of the Java JGit library. The issue with NGit is that the classes and methods required to replace the connection plumbing are internal or private. There is nothing preventing us from adding a custom type to interact with TFS, but we cannot inherit from the base classes that implement much of the lower level logic. And even if we could inherit from those base classes there are several required nested classes and utility types that cannot be accessed due to protection.
Just like with the approach using "git.exe" we could setup "alternative credentials" in TFS. In this case we could use NGit without modification, and pass a username and password from the build definition into NGit. But it would be simpler to just use Git outside of TFS to copy the repository.
A similar approach, although not ideal, is to modify the accessibility in NGit to allow us to replace the connection plumbing. NGit is published on GitHub and on cursory review the project appears to be relatively stable. There is no technical reason we cannot simply do a global replacement of access modifiers. As NGit is updated, the process can be repeated. The risk of creating problems modifying the accessibility is far less than recreating the low level functionality that would be needed to use the library in its current form.
A fourth approach would be to use a Git hook to trigger the mirroring. However, Team Foundation Service does not support Git hooks [3].
5.0. Solution
Of the possible approaches, the most straight forward solution is to:
- Create a custom build activity that mirrors the TFS Git repository being built to public Git services
- Use the NGit library to clone and push a mirrored copy of the Git repository being built to public Git services
- Modify accessibility in the NGit library to allow us to replace the connection plumbing
- Replace the NGit connection plumbing using the same approach that TFS uses to replace the connection plumbing of LibGit2Sharp so the library can connect to TFS
5.1. Prototype
Creating a build activity is straight forward and well documented. But we quickly run into a problem figuring out how to use the TFS API with a Git repository. There is little to no documentation on how to work with Git from within a build activity. We cannot use Workstation.Current.GetLocalWorkspaceInfo(... to create a reference to the server, and the build definition does not have the same set of variables as a traditional TFVC build template.
The build activity context ends up providing us access to the TFS server. See Figure 5-1.
Figure 5-1. TFS Api in a Git Repository Build Activity.
...
IBuildDetail BuildDetail = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<IBuildDetail>(Context, WellKnownEnvironmentVariables.BuildDetail);
IBuildDefinitionSourceProvider DefaultSourceProvider = BuildDetail.BuildDefinition.GetDefaultSourceProvider();
string RepositoryUrl = "";
string RepositoryName = "";
if (BuildSourceProviders.IsGit(DefaultSourceProvider.Name) == true)
{
RepositoryUrl = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryUrl);
RepositoryName = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryName);
}
string SourcesDirectory = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<string>(Context, WellKnownEnvironmentVariables.SourcesDirectory);
TfsTeamProjectCollection TFSServer = BuildDetail.BuildServer.TeamProjectCollection;
string TeamProject = BuildDetail.TeamProject;
if (TeamProject != RepositoryName)
{
RepositoryUrl = TFSServer.Uri.ToString() + "/" + TeamProject + "/_git/" + RepositoryName;
}
...
NGit handles connections to a Git repository through a "Transport Protocol".
Effectively NGit parses a URI and finds a matching protocol type. The protocol type then creates a transport object that knows how to work with the repository.
The NGit HTTP transport creates a simple connection object that uses a HttpWebRequest to make calls to the Git server. A new connection object is created for each call.
We can implement our own transport by creating a custom transport type that inherits from the NGit HTTP transport type NGit.Transport.TransportHttp. We can then override the method that creates connection objects and return our own connection type. Our connection type can inherit the NGit HTTP connection type Sharpen.URLConnection and replace the use of HttpWebRequest with HttpClient. This allows us to then pass in a VssHttpMessageHandler to take care of authentication in the same way the TFS does with LibGit2Sharp.
Figure 5-2 shows a proof of concept build activity. A list of output Git repositories are passed in through the OutputRepositoryUrls parameter. The
URLs are in the format of https://USERNAME:PASSWORD@domain.org/abc/efg.git.
The general workflow of the build activity is:
- Create a reference to the current server and build
- Clear loaded NGit transport protocols
- Add custom transport protocol (Figure 5-3)
- Create a "bare clone" of the TFS Git repository being built
- Remove the custom transport protocol and restore the originally loaded NGit transport protocols
- Push a mirrored copy of the "bare clone" to each of the output repositories
- Delete the "bare clone" repository copy
Figure 5-3 shows the custom transport protocol. The protocol returns a custom transport type (Figure 5-4), which in turns returns a custom connection object (Figure 5-5).
The custom connection object uses HttpClient in place of HttpWebRequest. This creates an issue with the output stream being closed by NGit before the message is sent. NGit uses HttpWebRequest.GetRequestStream() to write output and then closes the stream after the message is written. This forces us to use a custom stream object (Figure 5-6) that cannot be closed by NGit.
Figure 5-2. "MirrorRepository" TFS Build Activity.
using Microsoft.TeamFoundation;
using Microsoft.TeamFoundation.Build.Activities.Extensions;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Build.Common;
using Microsoft.TeamFoundation.Build.Workflow.Activities;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Git.Client;
using Microsoft.TeamFoundation.Git.Common;
using Microsoft.TeamFoundation.VersionControl.Client;
using Microsoft.VisualStudio.Services.Common;
using System;
using System.Activities;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.ExceptionServices;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git.Activities
{
[Microsoft.TeamFoundation.Build.Client.BuildActivity(
Microsoft.TeamFoundation.Build.Client.HostEnvironmentOption.All)]
[Microsoft.TeamFoundation.Build.Client.BuildExtension(
Microsoft.TeamFoundation.Build.Client.HostEnvironmentOption.All)]
public class MirrorRepository : System.Activities.AsyncCodeActivity
{
#region Private Variables
#endregion
#region Constructors
public MirrorRepository()
{
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
[System.Activities.RequiredArgument]
public System.Activities.InArgument<IEnumerable<string>>
OutputRepositoryUrls { get; set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
#endregion
#region Private Methods
protected sealed override IAsyncResult BeginExecute(
AsyncCodeActivityContext Context, AsyncCallback Callback, object State)
{
System.Threading.Tasks.Task ExecutionTask = this.ExecuteAsync(Context);
System.Threading.Tasks.TaskCompletionSource<object> TCS =
new System.Threading.Tasks.TaskCompletionSource<object>(State);
ExecutionTask.ContinueWith(T =>
{
if (T.IsFaulted)
{
TCS.TrySetException(T.Exception.InnerExceptions);
}
else if (T.IsCanceled)
{
TCS.TrySetCanceled();
}
else
{
TCS.TrySetResult(null);
}
if (Callback != null)
{
Callback(TCS.Task);
}
});
return TCS.Task;
}
protected sealed override void EndExecute(AsyncCodeActivityContext Context, IAsyncResult Result)
{
System.Threading.Tasks.Task ExecutionTask = (Task)Result;
}
protected async Task ExecuteAsync(AsyncCodeActivityContext Context)
{
List<System.IO.DirectoryInfo> CleanupDirectories = new List<System.IO.DirectoryInfo>();
try
{
List<Uri> OutputRepositories = new List<Uri>();
foreach (string OutputRepositoryUrl in this.OutputRepositoryUrls.Get(Context))
{
try
{
if (String.IsNullOrWhiteSpace(OutputRepositoryUrl) == false)
{
OutputRepositories.Add(new Uri(OutputRepositoryUrl));
}
}
catch { }
}
if (OutputRepositories.Count() > 0)
{
IBuildDetail BuildDetail = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<IBuildDetail>(Context, WellKnownEnvironmentVariables.BuildDetail);
IBuildDefinitionSourceProvider DefaultSourceProvider =
BuildDetail.BuildDefinition.GetDefaultSourceProvider();
string RepositoryUrl = "";
string RepositoryName = "";
if (BuildSourceProviders.IsGit(DefaultSourceProvider.Name) == true)
{
RepositoryUrl = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryUrl);
RepositoryName = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryName);
}
string SourcesDirectory = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<string>(Context, WellKnownEnvironmentVariables.SourcesDirectory);
TfsTeamProjectCollection TFSServer = BuildDetail.BuildServer.TeamProjectCollection;
string TeamProject = BuildDetail.TeamProject;
if (TeamProject != RepositoryName)
{
RepositoryUrl = TFSServer.Uri.ToString() + "/" + TeamProject + "/_git/" + RepositoryName;
}
List<NGit.Transport.TransportProtocol> NGitTransportProtocols =
new List<NGit.Transport.TransportProtocol>();
try
{
NGitTransportProtocols.AddRange(NGit.Transport.Transport.GetTransportProtocols());
}
catch { }
foreach (NGit.Transport.TransportProtocol T in NGitTransportProtocols)
{
NGit.Transport.Transport.Unregister(T);
}
NGit.Transport.TransportProtocol TFSTransportProtocol =
new Adapt.Build.Server.TFS.Git.TFSTransportProtocol(
new Func<Uri, VssCredentials>(
(EndPoint) =>
{
VssCredentials vssCredential = VssCredentials.LoadCachedCredentials(
CredentialsStorageRegistryKeywords.Build,
EndPoint,
false,
CredentialPromptType.DoNotPrompt);
return vssCredential;
}));
NGit.Transport.Transport.Register(TFSTransportProtocol);
Uri SourceRepositoryUrl = new Uri(RepositoryUrl, UriKind.RelativeOrAbsolute);
Context.TrackBuildMessage("Source Repository Url: " +
RepositoryUrl, BuildMessageImportance.High);
System.IO.DirectoryInfo SourceRepositoryGitDirectory = new System.IO.DirectoryInfo(
System.IO.Path.Combine(System.IO.Path.GetTempPath(),
Guid.NewGuid().ToString() + "\\"));
try
{
if (SourceRepositoryGitDirectory.Exists == false)
{
SourceRepositoryGitDirectory.Create();
}
}
catch { }
CleanupDirectories.Add(SourceRepositoryGitDirectory);
bool IsCloneSuccess = false;
try
{
NGit.Api.CloneCommand Command = new NGit.Api.CloneCommand();
Command.SetURI(SourceRepositoryUrl.ToString());
Command.SetDirectory(SourceRepositoryGitDirectory.FullName);
Command.SetCloneAllBranches(true);
Command.SetNoCheckout(true);
Command.SetBare(true);
Command.Call();
IsCloneSuccess = true;
}
catch (Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
NGit.Transport.Transport.Unregister(TFSTransportProtocol);
foreach (NGit.Transport.TransportProtocol T in NGitTransportProtocols)
{
NGit.Transport.Transport.Register(T);
}
if (IsCloneSuccess == true)
{
foreach (Uri OutputRepository in OutputRepositories)
{
string OutputRepositoryLogSafeUrl = OutputRepository.ToString();
if (OutputRepositoryLogSafeUrl.Contains('@') == true)
{
OutputRepositoryLogSafeUrl = OutputRepository.Scheme +
"://" + OutputRepositoryLogSafeUrl.Split('@')[1];
}
Context.TrackBuildMessage("Output Repository Url: " +
OutputRepositoryLogSafeUrl, BuildMessageImportance.High);
MirrorRepository.TryToGitPushMirror(SourceRepositoryGitDirectory, OutputRepository, Context);
}
}
}
else
{
Context.TrackBuildError("Output Repository Url is not provided.");
}
}
catch(Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
foreach (System.IO.DirectoryInfo CleanupDirectory in CleanupDirectories)
{
try
{
CleanupDirectory.Delete();
}
catch { }
}
}
private static IEnumerable<NGit.Transport.PushResult> TryToGitPushMirror(
System.IO.DirectoryInfo SourceRepositoryGitDirectory,
Uri OutputRepositoryUrl,
AsyncCodeActivityContext Context)
{
List<NGit.Transport.PushResult> Results = new List<NGit.Transport.PushResult>();
try
{
SourceRepositoryGitDirectory.Refresh();
if (SourceRepositoryGitDirectory.Exists == true)
{
NGit.Repository SourceRepository =
new NGit.Storage.File.FileRepository(SourceRepositoryGitDirectory.FullName);
try
{
SourceRepository.Create();
}
catch { }
NGit.Transport.URIish OutputRepositoryURIish = new NGit.Transport.URIish(OutputRepositoryUrl);
NGit.Storage.File.FileBasedConfig SourceRepositoryConfig =
((NGit.Storage.File.FileBasedConfig)SourceRepository.GetConfig());
string ConfigurationFile = SourceRepositoryConfig.GetFile();
string SourceRepositoryConfigFileContent = System.IO.File.ReadAllText(ConfigurationFile);
string OutputRepositoryRemoteName =
System.Guid.NewGuid().ToString().Replace("-", "");
NGit.Transport.RemoteConfig OutputRepositoryRemoteConfig =
new NGit.Transport.RemoteConfig(SourceRepositoryConfig, OutputRepositoryRemoteName);
OutputRepositoryRemoteConfig.IsMirror = true;
OutputRepositoryRemoteConfig.AddURI(OutputRepositoryURIish);
OutputRepositoryRemoteConfig.Update(SourceRepositoryConfig);
SourceRepositoryConfig.Save();
try
{
NGit.Api.Git SourceRepositoryGit = new NGit.Api.Git(SourceRepository);
NGit.Api.PushCommand PushMirrorCommand = SourceRepositoryGit.Push();
PushMirrorCommand.SetRemote(OutputRepositoryRemoteName);
PushMirrorCommand.SetForce(true);
PushMirrorCommand.Add("+refs/*:refs/*");
Sharpen.Iterable<NGit.Transport.PushResult> CommandResults =
PushMirrorCommand.Call();
try
{
Results.AddRange(CommandResults);
}
catch { }
}
catch (Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
System.IO.File.WriteAllText(ConfigurationFile, SourceRepositoryConfigFileContent);
}
}
catch (Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
return Results;
}
#endregion
#region Overridden Methods
#endregion
}
}
Figure 5-3. "TFSTransportProtocol" NGit Transport Protocol.
using Microsoft.VisualStudio.Services.Common;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class TFSTransportProtocol : NGit.Transport.TransportProtocol
{
#region Private Variables
#endregion
#region Constructors
public TFSTransportProtocol(Func<Uri,VssCredentials> GetCredentials)
{
this.GetCredentials = GetCredentials;
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
protected Func<Uri,VssCredentials> GetCredentials { get; private set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
public override string GetName()
{
return this.GetType().Name;
}
public override ICollection<string> GetSchemes()
{
List<string> Schema = new List<string>();
Schema.Add("http");
Schema.Add("https");
return Schema;
}
public override ICollection<NGit.Transport.TransportProtocol.URIishField> GetRequiredFields()
{
List<NGit.Transport.TransportProtocol.URIishField> Fields =
new List<NGit.Transport.TransportProtocol.URIishField>();
Fields.Add(NGit.Transport.TransportProtocol.URIishField.HOST);
Fields.Add(NGit.Transport.TransportProtocol.URIishField.PATH);
return Fields;
}
public override ICollection<NGit.Transport.TransportProtocol.URIishField> GetOptionalFields()
{
List<NGit.Transport.TransportProtocol.URIishField> Fields =
new List<NGit.Transport.TransportProtocol.URIishField>();
Fields.Add(NGit.Transport.TransportProtocol.URIishField.USER);
Fields.Add(NGit.Transport.TransportProtocol.URIishField.PASS);
Fields.Add(NGit.Transport.TransportProtocol.URIishField.PORT);
return Fields;
}
public override int GetDefaultPort()
{
return 80;
}
public override NGit.Transport.Transport Open(NGit.Transport.URIish RemoteRepositoryUrl)
{
return new TFSTransport(
RemoteRepositoryUrl,
new Func<Uri, System.Net.Http.HttpMessageHandler>((EndPoint) =>
{
return GetVssHttpMessageHandler(this.GetCredentials.Invoke(EndPoint));
}));
}
public override NGit.Transport.Transport Open(NGit.Transport.URIish RemoteRepositoryUrl,
NGit.Repository LocalRepository, string RemoteName)
{
return new TFSTransport(
LocalRepository,
RemoteRepositoryUrl,
new Func<Uri, System.Net.Http.HttpMessageHandler>((EndPoint) =>
{
return GetVssHttpMessageHandler(this.GetCredentials.Invoke(EndPoint));
}));
}
#endregion
#region Private Methods
protected static VssHttpMessageHandler GetVssHttpMessageHandler(VssCredentials Credentials)
{
return new VssHttpMessageHandler(Credentials,
new VssHttpRequestSettings() { ExpectContinue = false });
}
#endregion
#region Overridden Methods
#endregion
}
}
Figure 5-4. "TFSTransport" NGit Transport.
using NGit;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class TFSTransport : NGit.Transport.TransportHttp
{
#region Private Variables
#endregion
#region Constructors
public TFSTransport(
NGit.Repository LocalRepository,
NGit.Transport.URIish RemoteRepositoryUrl,
Func<Uri, System.Net.Http.HttpMessageHandler> GetHttpClientHandlerInstance = null)
: base(LocalRepository, RemoteRepositoryUrl)
{
this.RemoteRepositoryUrl = RemoteRepositoryUrl;
this.GetHttpClientHandlerInstance = GetHttpClientHandlerInstance;
}
public TFSTransport(
NGit.Transport.URIish RemoteRepositoryUrl,
Func<Uri, System.Net.Http.HttpMessageHandler> GetHttpClientHandlerInstance = null)
: base(RemoteRepositoryUrl)
{
this.RemoteRepositoryUrl = RemoteRepositoryUrl;
this.GetHttpClientHandlerInstance = GetHttpClientHandlerInstance;
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
protected NGit.Transport.URIish RemoteRepositoryUrl { get; private set; }
public NGit.Repository LocalRepository
{
get
{
return base.local;
}
}
protected Func<Uri, System.Net.Http.HttpMessageHandler>
GetHttpClientHandlerInstance { get; private set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
#endregion
#region Private Methods
public override Sharpen.HttpURLConnection HttpOpen(string method, Uri u)
{
System.Net.Http.HttpMessageHandler ClientHandler = null;
if (this.GetHttpClientHandlerInstance != null)
{
try
{
ClientHandler = this.GetHttpClientHandlerInstance.Invoke(u);
}
catch { }
}
Sharpen.HttpURLConnection Connection = new HttpTransportConnection(u, ClientHandler);
Connection.SetRequestMethod(method);
Connection.SetUseCaches(false);
Connection.SetRequestProperty(NGit.Util.HttpSupport.HDR_ACCEPT_ENCODING,
NGit.Util.HttpSupport.ENCODING_GZIP);
Connection.SetRequestProperty(NGit.Util.HttpSupport.HDR_PRAGMA, "no-cache");
return Connection;
}
#endregion
#region Overridden Methods
#endregion
}
}
Figure 5-5. "HttpTransportConnection" NGit Http Connection.
using NGit;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class HttpTransportConnection : Sharpen.HttpURLConnection, IDisposable
{
#region Private Variables
private bool disposed = false;
#endregion
#region Constructors
public HttpTransportConnection(
Uri EndPoint,
System.Net.Http.HttpMessageHandler ClientHandler = null)
: base(EndPoint, null)
{
this.EndPoint = EndPoint;
this.ClientHandler = ClientHandler;
this.Request = new System.Net.Http.HttpRequestMessage(System.Net.Http.HttpMethod.Get, EndPoint);
this.RequestContentStream = new TFSTransportStream();
this.Request.Content = new System.Net.Http.StreamContent(this.RequestContentStream);
if (this.ClientHandler != null)
{
this.Client = new System.Net.Http.HttpClient(this.ClientHandler);
}
else
{
this.Client = new System.Net.Http.HttpClient();
}
this.Response = new Lazy<System.Net.Http.HttpResponseMessage>(() =>
{
if(this.Request.Method != System.Net.Http.HttpMethod.Post &&
this.Request.Method != System.Net.Http.HttpMethod.Put )
{
this.Request.Content = null;
}
this.RequestContentStream.Position = 0;
System.Net.Http.HttpResponseMessage Response = this.Client.SendAsync(this.Request).Result;
this.RequestContentStream.IsComplete = true;
this.RequestContentStream.Close();
return Response;
}, true);
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
private Uri EndPoint { get; set; }
private System.Net.Http.HttpClient Client { get; set; }
private System.Net.Http.HttpMessageHandler ClientHandler { get; set; }
private System.Net.Http.HttpRequestMessage Request { get; set; }
private TFSTransportStream RequestContentStream { get; set; }
private Lazy<System.Net.Http.HttpResponseMessage> Response { get; set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
public override void SetUseCaches(bool u)
{
}
public override void SetRequestMethod(string method)
{
this.Request.Method = new System.Net.Http.HttpMethod(method);
}
public override string GetRequestMethod()
{
return this.Request.Method.Method;
}
public override void SetInstanceFollowRedirects(bool redirects)
{
}
public override void SetDoOutput(bool dooutput)
{
}
public override void SetFixedLengthStreamingMode(int len)
{
}
public override void SetChunkedStreamingMode(int n)
{
}
public override void SetRequestProperty(string key, string value)
{
switch(key.ToLowerInvariant())
{
case "content-encoding":
this.Request.Content.Headers.ContentEncoding.Add(value);
break;
case "content-length":
long ContentLength = 0;
if (long.TryParse(value, out ContentLength) == true)
{
this.Request.Content.Headers.ContentLength = ContentLength;
}
break;
case "content-type":
this.Request.Content.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue(value);
break;
case "transfer-encoding":
this.Request.Content.Headers.Add("transfer-encoding", value);
break;
default:
this.Request.Headers.Add(key, value);
break;
}
}
public override string GetResponseMessage()
{
return this.Response.Value.ReasonPhrase;
}
public override void SetConnectTimeout(int ms)
{
}
public override void SetReadTimeout(int ms)
{
}
public override Sharpen.InputStream GetInputStream()
{
return this.Response.Value.Content.ReadAsStreamAsync().Result;
}
public override Sharpen.OutputStream GetOutputStream()
{
return this.RequestContentStream;
}
public override string GetHeaderField(string header)
{
string Value = "";
switch (header.ToLowerInvariant())
{
case "content-encoding":
Value = Value = String.Join(", ",
this.Response.Value.Content.Headers.ContentEncoding);
break;
case "content-length":
Value = this.Response.Value.Content.Headers.ContentLength.ToString();
break;
case "content-type":
Value = this.Response.Value.Content.Headers.ContentType.MediaType;
break;
default:
if (this.Response.Value.Headers.Select(x => x.Key).Contains(header) == true)
{
Value = String.Join(", ", this.Response.Value.Headers.GetValues(header));
}
else if (this.Response.Value.Content.Headers.Select(x => x.Key).Contains(header) == true)
{
Value = String.Join(", ", this.Response.Value.Content.Headers.GetValues(header));
}
break;
}
return Value;
}
public override string GetContentType()
{
return this.Response.Value.Content.Headers.ContentType.MediaType;
}
public override int GetContentLength()
{
return (int)this.Response.Value.Content.Headers.ContentLength.Value;
}
public override int GetResponseCode()
{
return (int)this.Response.Value.StatusCode;
}
public override Uri GetURL()
{
return this.EndPoint;
}
public virtual void OnDisposing()
{
try
{
this.ClientHandler.Dispose();
this.ClientHandler = null;
}
catch { }
try
{
this.Client.Dispose();
this.Client = null;
}
catch { }
}
#endregion
#region Private Methods
public void Dispose()
{
this.Dispose(true);
GC.SuppressFinalize(this);
}
protected void Dispose(bool disposing)
{
if (!this.disposed)
{
if (disposing)
{
try
{
this.OnDisposing();
}
catch { }
}
this.disposed = true;
}
}
~HttpTransportConnection()
{
this.Dispose(true);
}
#endregion
#region Overridden Methods
#endregion
}
}
Figure 5-6. "TFSTransportStream" Stream.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class TFSTransportStream : System.IO.MemoryStream
{
#region Private Variables
#endregion
#region Constructors
public TFSTransportStream()
{
this.IsComplete = false;
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
public bool IsComplete { get; set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
public override void Close()
{
if (this.IsComplete == true)
{
base.Close();
}
}
#endregion
#region Private Methods
#endregion
#region Overridden Methods
#endregion
}
}
5.2. Download
A complete proof of concept Visual Studio solution is available for download under Resources. A modified copy of NGit,
which opens accessibility to allow types to be inherited, is included in the sample project. The modified copy will be released as a NuGet package with the source code published.
5.3. Known Issues
1. Continuous Integration Build Trigger
Team Foundation Service’s support for Git repositories is limited. There appears to be an issue using a custom Git build process template with an automated trigger.
The default template works with continuous integration triggers, but custom templates throw an exception. The exception appears to be related to the build initialization.
The exception is thrown even with an unaltered copy of the default template.
TF215097: An error occurred while initializing a build for build definition
Exception Message: TF10159:
The label name 'G:refs/heads/master:CB8ED4F70E06E1519FBF205C0D7254244DD7AA0B' is not supported.
The exception is not thrown when a build is queued manually.
6.0. Due Diligence and Critical Risks
Possible issues with the prototype include:
- The author has a limited knowledge of the inner workings of Git. It is possible that the mirroring implementation may not work in all circumstances.
- The stream object used to prevent NGit from closing an output stream before it is sent, simply inherits from
System.IO.MemoryStream
and overrides the Close() method. This simple implementation may cause issues. - The prototype has not been extensively field tested across different production environments.
7.0. End Notes
- See:
Microsoft.TeamFoundation.Build.Activities.Git
namespace in Microsoft.TeamFoundation.Build.Activities.dll (Version 12.0.0.0) - See:
http://tfs.visualstudio.com/en-us/learn/hosted-build-controller-in-vs.aspx
- See:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/84fea5b9-de95-4157-bb70-5333f86a8eb4/git-commit-hook-stream-available
