Summary
This is an application deployment system for ASP.NET, including code-based or non-code publishing of DEV to QA, QA to Staging and Staging to Production. Asynchronous technique is used in this project to handle long-running processes such as file copy, status update, application warm-up and simultaneous publishing to multiple servers.
Introduction
When building an application deployment system, we expect it to be able to manage different stages of deployment of DEV to QA, QA to Staging and Staging to Production, in an efficient, robust and convenient way.
Additionally, it is expected to have the following features:
- Status of the deployment process is reported to the user when there is an update.
- Application build can be delivered simultaneously to multiple production servers within a cluster.
- Application warm-up after deployment.
- File copy should be robust and efficient.
- Non-code deployment is convenient and requires no build.
- Certain level of logging is required to audit events.
- Administration is done via a web interface so that it can be simply accessed by a browser.
Asynchronous Request Handling
The key is to utilize asynchronous techniques in .NET to handle long-running processes such as file copying, status update, application warm-up and simultaneous publishing to multiple servers.
QAPublish, StagingPublish and ProductionPublish are three classes handling the three stages of deployment. They all implement an interface IAsyncRequest. AsyncRequestState is a wrapper class that is used to hold HttpContext, a callback function and extra data passed to IAsyncRequest members. In the case of ProductionPublish, HttpContext.Cache is employed as a temporary storage for deployment status and application warm-up result.
public interface IAsyncRequest
{
void ProcessRequest();
AsyncRequestState AsyncRequestState{get;}
}public class ProductionPublish : IAsyncRequest
{
private AsyncRequestState _asyncRequestState;
private static string lockString = string.Empty;
public ProductionPublish(AsyncRequestState ars)
{
_asyncRequestState = ars;
}
private void UpdateStatus(string server, PublishStatus status)
{
lock(lockString)
{
Hashtable ht =
_asyncRequestState._ctx.Cache[SR.PubStatusSessionName] as Hashtable;
if (ht.ContainsKey(server))
{
ht[server] = status;
}
else
{
ht.Add(server, status);
}
_asyncRequestState._ctx.Cache[SR.PubStatusSessionName] = ht;
}
}
private void CheckStatus()
{
lock(lockString)
{
Hashtable ht =
_asyncRequestState._ctx.Cache[SR.PubStatusSessionName] as Hashtable;
if (ht == null)
{
ht = new Hashtable();
_asyncRequestState._ctx.Cache[SR.PubStatusSessionName] = ht;
}
}
}
private PublishStatus GetStatus(string server)
{
Hashtable ht =
_asyncRequestState._ctx.Cache[SR.PubStatusSessionName] as Hashtable;
if (ht.ContainsKey(server))
{
return (PublishStatus)ht[server];
}
return null;
}
AsyncRequestState IAsyncRequest.AsyncRequestState
{
get
{
return _asyncRequestState;
}
}
void IAsyncRequest.ProcessRequest()
{
Process proc = null;
PublishStatus st;
string output;
try
{
string server = (string)((Pair)_asyncRequestState._extraData).First;
ProcessStartInfo procInfo = new ProcessStartInfo();
procInfo.UseShellExecute = true;
procInfo.WorkingDirectory = Settings.ProdPubBatchPath;
procInfo.FileName = Settings.ProdPubBatchFile;
procInfo.Arguments = string.Format("{0} {1}",
server, Settings.StagingRootFolder);
procInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
CheckStatus();
UpdateStatus(server, new PublishStatus(server, SR.Copying, null));
System.Diagnostics.Trace.WriteLine(SR.CopyStartedLogMsg(server,
Utility.Now()));
proc = Process.Start(procInfo);
proc.WaitForExit();
System.Diagnostics.Trace.WriteLine(SR.CopyEndedLogMsg(server,
Utility.Now()));
UpdateStatus(server, new PublishStatus(server,
SR.Copied, SR.WarmingUp));
System.Diagnostics.Trace.WriteLine(SR.FirstLoadStartedLogMsg(server,
Utility.Now()));
output = Utility.MakeWebRequest(server);
System.Diagnostics.Trace.WriteLine(SR.FirstLoadEndedLogMsg(server,
Utility.Now()));
st = GetStatus(server);
if (output != null)
{
st.WarmUpResult = output;
st.WarmUpStatus = SR.WarmedUp;
}
else
{
st.WarmUpResult = string.Empty;
st.WarmUpStatus = SR.Timeout;
}
UpdateStatus(server, st);
st = GetStatus(server);
st.LoadStatus = SR.Loading;
UpdateStatus(server, st);
System.Diagnostics.Trace.WriteLine(
SR.SecondLoadStartedLogMsg(server, Utility.Now()));
output = Utility.MakeWebRequest(server);
System.Diagnostics.Trace.WriteLine(
SR.SecondLoadEndedLogMsg(server, Utility.Now()));
st = GetStatus(server);
if (output != null)
{
st.LoadResult = output;
st.LoadStatus = SR.Loaded;
}
else
{
st.LoadResult = string.Empty;
st.LoadStatus = SR.Timeout;
}
UpdateStatus(server, st);
_asyncRequestState.CompleteRequest();
}
catch (Exception ex)
{
}
}
}
The following is the code to start the asynchronous publishing process in the web-based administration interface:
AsyncRequestState reqState =
new AsyncRequestState(Context, null, new Pair(srv, null));
ProductionPublish ar = new ProductionPublish(reqState);
Publisher pub = new Publisher();
pub.BeginProcessRequest(ar);
File Copy Engine � Robocopy
It is not necessary to develop a new file copy program since one is already available in the Windows Resource Kit --Robocopy. This powerful command-line tool can accomplish a variety of scripted copying tasks, including large data migrations and server consolidations. It can be configured to filter files, manipulate file attributes, and do proper logging in the copying process. Without reinventing the wheels, this tool can be readily adapted as the file copy engine. It can be downloaded from various Internet locations by Googling �robocopy�. A manual is available here.
We will be impersonating an account which will have sufficient rights to run Robocopy as an external process and access the internal networks where all involved servers reside, to the web-enabled deployment system.
Robocopy is used via batch files which are scripted to deal with various stages of file copying and different pre and post deployment scenarios.
A sample batch file is included in the project.
Site Refresh / Non-code Publishing
In addition to normal daily or nightly builds that are deployed to QA or Staging environments on a certain schedule, quick fixes are sometimes required to be deployed without rerunning a build for non-code files such as aspx, ascs, js, css and so on. I will refer this as a site refresh.
In this system, I included a site refresh managing interface where multiple non-code files from DEV can be added to the publish-queue and deployed to QA or Staging with proper Visual SourceSafe labeling on these files. (You can remove VSS access from the system if your DEV build doesn�t do VSS labeling.)
VSS access is achieved via methods in a SourceSafe helper class which is taken from Microsoft BuildIt tool with some minor modifications. Please refer to BuildIt.
Configurations
The system uses web.config to store application settings. It is important that certain configurations have to be made in order to get the application running.
The following appSettings section in web.config should be fairly self-explanatory:
<appSettings>
-->
<add key="RootPath" value="C:\Projects\NPublisher\Web\"/>
-->
<add key="AllowExtensions"
value="|.aspx|.ascx|.xml|.ico|.config|.js|.txt|.html|.css|"/>
-->
-->
<add key="VSSUsername" value="Username"/>
<add key="VSSPassword" value="Password"/>
<add key="IniFilePath" value="\\VssServer\DBFolder"/>
-->
<add key="SrcVSSRootFolder" value="$/NPublisher/Web/"/>
-->
<add key="SrcFileRootFolder" value="C:\Projects\NPublisher\Web\"/>
-->
-->
<add key="QAServer" value="QAServer"/>
<add key="QARootFolder" value="\\QAServer\WebRoot\"/>
-->
<add key="QAPubBatchFile" value="Publisher.bat" />
<add key="QAPubBatchPath" value="C:\Batch\QA\" />
-->
<add key="StagingServer" value="StagingServer"/>
<add key="StagingRootFolder" value="\\StagingServer\WebRoot\"/>
-->
<add key="StagingPubBatchFile" value="Publisher.bat" />
<add key="StagingPubBatchPath" value="C:\Batch\Staging\" />
-->
<add key="ProdServers"
value="ProdServer1|ProdServer2|ProdServer3|ProdServer4|ProdServer5" />
<add key="ProdPubBatchFile" value="Publisher.bat" />
<add key="ProdPubBatchPath" value="C:\Batch\Production\" />
-->
<add key="WarmUpUrl" value="http://{0}/Warmup.aspx" />
<add key="WarmUpTimeout" value="100000" />
-->
</appSettings>
Also, make sure the account ASP.NET is running under has sufficient permissions to write tracing logs and access files locally or on remote network shares.
You can impersonate a domain account in the application by adding the following line in the web.config:
<identity impersonate="true" userName="username" password="password" />
Latest Release
Please get the latest release at NPublisher GotDotNet Workplace.
References
