We often got requirements from client where they asked us to develop promo campaign that they believe can help them to get the market. In many cases they design such campaign based on their hypothesis and market analysis report. Sometime their hypothesis works but some time it fails badly. One way to avoid such failure is to test the hypothesis with some real user. A/B testing (also known as split testing) does exactly the same. You compare two campaign by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that produce higher % of success determines which campaign you should stick with.
As you can imagine this kind of setup (traffic routing based on some custom rule) requires a lot of works . Azure offers this feature just out of the box. If your site is hosted as PaaS then login to your azure dashboard, tweak some settings and you are done! Azure will automatically load balance a percentage of the traffic going to your site between production and your designated slot based on your configuration. In azure this awesome feature was called "Testing in Production" and now the name has slightly changed to "Traffic Routing".
Lets say, our client came up with two campaign offer and they would like to figure out which one is creating positive impact on users. They would like to route 10% traffic to the first campaign and another 10% to the second campaign. The rest 80% traffic would go the the older site.
To have this setup lets first login to your azure account and select the particular website for which you would like to redirect the visitors. Clicking the settings tab will open the Settings blade as shown in the following figure.
Figure : Settings Blade[/caption]
Click the Traffic Routing to open up a new blade to configure the traffic. Choose a deployment slot and set the traffic.
Figure : Traffic Routing Blade
Configure all the slot from this window. You can also create new slot by clicking Add Slot option from the top blade.
Figure : Traffic configuring for different slot
Here, we have setup 10% traffic for Campaign-Offer-1 and Campaign-Offer-2. The rest 80% traffic will go to the actual site. Then we can hook up Application Insights, New Relic or some other event/diagnostics system to measure the difference in user reaction between this two campaign. The one that produce most positive response will be replace with the old site.
Heads Up:
- To ensure a seamless experience.to your user, make sure you write enough code to manage their authentication and session information.
- If you need to perform stress trace, keep in mind that they share the same resources as your production site. So doing such test could affect your production site! On such case move the test site/slot to a separate resource group.
Wrapping Up:
By default 100% of the traffic goes to the Production slot but there could be situation where you might want to redirect some of your customers to a different version of your site and want to figure out if the new changes bringing any positive impact on your site. Situation like that can be easily handled with the feature named "Traffic Routing".
Good Read :
- http://www.hanselman.com/blog/ABTestingAndTestingInProductionWithAzureWebApps.aspx
