Amazon AWSDeploy To Provision Multiple Websites

Introduction

Recently I've been on a project where we are implementing BlueGreen deployments using Amazon CloudFormation, and came across a number of unanswered questions:

• How do you deploy multiple applications to a single instance and support auto-scaling?
• How do you deploy non-website applications using AWSDeploy?
• How can you pass in environmental config during provisioning?

The official answer to the first 2 questions appears to be that you can't without additional scripts/tools. The third (with some caveats) is that you need to bake them into your package prior to your deployment.

This seems frustrating for such common real world use cases, but luckily there is a solution which requires minimal effort.

First Some Background

CloudFormation is Amazon's offering for provisioning a collection of AWS resources on the amazon infrastructure through declarative templates. The collection of resources is known as a stack and can be used for reliable and repeatable infrastructure deployments.

AWSDeploy is the tool which can be used with the Microsoft stack to provision CloudFormation templates and to deploy web applications onto these provisioned resources. This uses Microsoft WebDeploy under the hood, and as such a WebDeploy package must be provided when provisioning a CloudFormation stack using the tool.

The WebDeploy package provided by the user is uploaded to AWS by the tool and stored in an S3 bucket. When the EC2 web server spins up, AWSDeploy installs the package on the web server as part of initialising the stack. By default any failures will result in a stack rollback.

Once a web application is provisioned via AWSDeploy, the stack will manage failover and auto-scaling scenarios automatically by redeploying the associated WebDepoy package stored on S3 to any new instances. In doing so it is able to manage resources hands-free and replicate perfect copies of your application on demand.

With Great Power Comes Great Limitations

As described above, AWSDeploy is incredibly useful for failover and scaling scenarios. Your web application can be replicated and auto-provisioned without manual interaction.

The downside is a number of limitations:

• Only 1 WebDeploy package can be associated with a web server instance
• No parameters can be passed when AWSDeploy installs the WebDeploy package
• As such WebDeploy parameterisation cannot be used for config transforms
• Environmental config is limited to a small set of pre-defined appSettings keys, which can be injected during provisioning

The impact is that to work with AWSDeploy you generally have to modify your build artifact for every deployment environment and bake in configuration settings. This is against Continuous Delivery principles and adds complexity to any deployment scripts.

An Idea

WebDeploy in it's own right is an extremely flexible and versatile tool. In a single WebDeploy package it's possible to deploy multiple websites, deploy directories of arbitrary files, execute commands, and change Acl permissions.

So, instead of baking environment configuration into the WebDeploy packages to support AWSDeploy, why not instead create a wrapper WebDeploy package and use an embedded RunCommand to pass environment variables to a child package during provisioning. In this way the original build artifact is unmodified and the full power of WebDeploy can be used to inject parameters during installation.

This approach has additional benefits in that it allows the install of multiple web applications via a single AWSDeploy package, as well as the ability to deploy Windows Services and Console applications during auto-provisioning. Additionally it's then relatively easy to cheery-pick applications for deployment to a particular instance with very little rework of deployment scripts.

The Problem

On creating my first wrapper package I immediately ran into a problem when provisioning. The AWSDeploy logs revealed the issue.

ERROR 10 AWSDeploymentHostManager.Tasks.UpdateAppVersionTask - Deploy failed, not iisApp.

It seems AWSDeploy expects the WebDeploy package to contain an iisApp within it's manifest. So I included a stub website in my WebDeploy package and tried again. Surprisingly I hit the same error!

After much investigation it turns out that when AWSDeploy executes the WebDeploy package, it passes the following parameter to the package. This is to enable customisation of the target IIS Website name during provisioning, and can be set via a AWSDeploy configuration setting.

-setParam:"IIS Web Application Name"="Default Web Site/"

The Solution

Knowing the parameter requirement, the solution is simple. Provided the WebDeploy package contains a parameters file with this parameter defined within, then AWSDeploy will successfully deploy the package regardless of it's contents. It also means that the WebDeploy package does not actually need to include an iisApp instance as the error first suggested.

Putting It All Together

First place any WebDeploy packages to be installed on the server instance in a folder called C:\Packages. Include any set-parameter configuration files for any environmental settings and config transforms.

Create a PowerShell install script called Install.ps1 and add this to the C:\Packages directory. The script will be executed on the server during provisioning. It will be used to deploy the WebDeploy packages in this folder on the server, and can be used to invoke explicit WebDeploy commands during install; such as to leverage environment specific parameterisation.

Set-Alias msdeploy "C:\Program Files\IIS\Microsoft Web Deploy V3\msdpeloy.exe"

msdeploy -verb:sync -source:package=WebsiteA.zip -dest:auto -setParamFile:WebSiteA-Params.xml
msdeploy -verb:sync -source:package=WebsiteB.zip -dest:auto -setParamFile:WebSiteB-Params.xml

Next create the parameters file for inclusion in the WebDeploy wrapper package as follows. Name the file parameters.xml and place it in a folder called C:\Build.

<parameters>
 <parameter name="IIS Web Application Name" defaultValue="Default Web Site/" tags="IisApp" />
</parameters>

Now create a WebDeploy manifest file for the wrapper package as follows. This will provision the contents of C:\Packages to the server instance and use the RunCommand provider to execute the Install.ps1 PowerShell script in the root of this directory. Name the file manifest.xml and place it in the C:\Build folder.

<siteManifest>
  <contentPath path="C:\Packages" />
  <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe
       -inputformat none -command &quot;Set-ExecutionPolicy Unrestricted -force&quot;" 
       waitInterval="10000" />
  <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe 
       -inputformat none -command C:\Packages\Install.ps1" 
       waitInterval="300000" />
</siteManifest>

Finally package up the WebDeploy wrapper package using the command line as follows, which should be executed from the C:\Build folder. The trick is the declareParamFile parameter and parameters.xml file which keeps AWSDeploy happy.

msdeploy -verb:sync -source:manifest=manifest.xml -dest:package=Deploy.zip
         -declareParamFile=parameters.xml

The WebDeploy package produced is now ready to be used with the AWSDeploy tool to provision 1 or more applications on your EC2 server. It is also fully compatible with auto-scaling and failover scenarios.

You can test the install locally by executing the following command in the C:\Build directory.

msdeploy -verb:sync -source:package=Deploy.zip -dest:auto

Further Enhancements

The solution minimises the customisation of deployment scripts for different environments, and results in a solution much less coupled to Amazon AWSDeploy. It also works around the limitations of AWSDeploy and provides a much greater level of flexibility in how instances can be configured to support auto-provisioning.

This is an improvement, but we are still having to build different artifacts per environment which is not ideal. A further enhancement would be to leverage AWS UserData from within the PowerShell script. UserData is configured during provisioning and made available to each server instance via a web service at a fixed address:

http://169.254.169.254/latest/user-data

UserData is persisted during failover & scaling, and could be used to either feed the PowerShell script entire set-parameter files dynamically, or provide flags to indicate to the PowerShell script which pre-deployed set-parameter file to use per environment when installing packages. In this way a single artifact could be used across all environments inline with best practice, as well as a single install and deployment script.