Introduction
We've reached a day when every business applications are built with a varieties
of complexity and grows with large volume of data. Over the time when data volume
grows or user base increase , there is a scope of decreasing the overall performance
of the application or even failed to deliver as expected. For this reason as a precaution
actual capability of the application must be measured in advance to avoid unexpected
situations. Now a days simple application which will serve single user consistently
is becoming antique. This is time when applications scalability and performance
plays a vital role in the software specification as well as architecture. And thus
the performance testing and analysis became handy. Goal of this article to provide
a quick and simple approach to configure load test for real scenario and analyze results. Through this article I will share my experience of implementing load testing
and analyzing results with visual studio 2010 ultimate.
Prerequisites
"load testing with visual studio 2010" goggling this phrase is more then enough
for anyone to find lots of fantastic articles about basic load testing setup. So
in this article we will not walk in that path, instead I will share some additional
info and share some configurations that will require to setup load testing for real
life and focus on the analyzing the results. But before moving on I recommend you
to learn basics of web test and load test if you are completely new to this kingdom.
Visual studio 2010 shipped with very handy testing tool, we are going to utilize
this tool to perform load testing. I recommend you to go through how to setup web test first, it will help you understand better.You can suggest any additional info to make
this post more helpful.
Load Testing Topology
Load testing involves generating traffic to a system and measuring how it responds.
Tools simulate heavy volumes of users taking various realistic scenarios in the
system and we will use visual studio 2010 for this purpose. There is two ways to
run load tests. Generate load from Visual Studio 2010 this will uses only 1 core/processor
with limited to 250 simulated users. Another way is load testing through test rig
which can simulate lots more users require much more more configuration in network
and hardware layer. To avoid complexity we will not focus on test rig, but some
day we will share this experiences too. The following diagram shows how a thorough
load test could be set up using visual studio 2010.
Load Testing
While doing load testing these are the considerations
- Understanding the environment of the application being tested
- A test plan defining user scenarios
- Tools to generate load and capture metrics
- Analysis of performance indicators
Visual studio 2010 can simulate approximately 1000 users per processor. we can generate
load from Visual Studio 2010 through 2 approach
- Default: uses only 1 core/processor and limited to 250 simulated users
- Test Rig: can use multiple core/processor and can simulate lots more users
It is recommended that performance testing database and application database shouldn't
on the same server otherwise while test controllers sample and save the load test
run results in the database on the application server it may impact the test results.
Visual studio load testing tool is consist of versatile concepts implemented and
msdn got all well explained, which could be way out of head for a starter, So rather
I listed few impotent concepts that are least required for carry out the load test.
- Think Time: the time spent by a user perusing a Web page, including viewing the
page and determining the next action. Think time does not include the time that
is required for a page to load.
- Ramp time: This property allows the increase number of user from one step to the
next to occur gradually rather than immediately
- Sample Rate: Time interval between collecting test data.For longer load tests, increasing
the sample rate reduces the amount of data that you collect.
- Run duration: Total time the test is expected to run.
- Warm up: the load test will increase the load gradually during the warm-up period
we will use above concepts are configured in our attached sample project. To simulate
real time request/response we can utilize mix that can control the percentage of
load that is distributed among tests, browser types, or network types in a load
test scenario.There are three types of mix
- Test mix: It can specify for the probability that a particular test will be randomly
selected by a virtual user during a load test run.
- Browser mix: It do not simulate the test in a stripped down browser. Test agents
includes a set of associated headers for the selected browsers and added to each
HTTP request before submit.
- Network mix: It simulate network bandwidth at the application layer. It does not
simulate latency.
Configure and Use Sample Project
Sample project consist of 2 projects
- TestingBox: test project consist of web test and performance test
- TestingBox-Demo: we project invoked by the load test
In order to use the sample project within your application, you need to configure theses parameters below according to your needs,
- Think Time: to simulate realistic wait times we set 3 seconds.
- Think Profile: Normal Distribution
- Step Ramp time: the user load would be increased from 10 to 20 users over a 10 second
period
- Sample Rate time: system will collect result data after each 10 second
- Run duration: 10 minutes which is calculated based on the step load distribution
- Warm up duration:load test will increase the load gradually within 5 seconds.
Another consideration is database.Visual Studio strictly requires SQL Server for
the load test store. If you don't have a SQL Server database, you cannot save test
run data. You can check the databse setting from Test (visual studio menu) > Manage
Test Controllers. A dialog will appear which allows set/reset of connection string
of test data store.
Analyze Results
The primary goal of performance testing is to study application performance under
normal load and heavy load for sustained durations or for short durations. In order
to know our applications capability, our load tetsing should be answer these questions
below
- Does web application function properly under given load?
- Does the page response times satisfy target requirements?
- Which one is the slowest page on web application during tests?
- Which page take longest time to download?
- How many simultaneous users can web application handle?
And in this section we will try to find these answers by analyzing graphs provided
by visual studio load testing.
At visual Studio 2010 several built-in graphs are provided. we can use the built-in
graphs as is or we can customize them. Additionally, we can create our own graphs.
Load test analyzer graphs display counters. Each graph follow a specific measurement
to plot a graph. The y-axis is not labeled with any specific range, but is instead
labeled with values from 0-100 that represent a percentage of the total range for
each counter. For example, for a counter with a range of 1000, a data point of 60
on the y-axis would correspond to a value of 600 for the counter. The horizontal
timeline x-axis for a collapsing graph has an origin of 00:00 and a range that covers
the entire test run. Average's are not plotted in graph.
We can analyze load test results by using the Load Test Analyzer in the following
ways:
- Monitor a load test in real time while it is running.
- Analyze a load test results
- Analyze results from previous load test(s).
Now lets try to find our answers from the graphs.
Does web application function properly under given load?
We can create an HTTP summary graph which will contain "Request time out","Passed
request","Failed Requests". This will provide time-based frequency of errors over
the duration of the test. Error rate is an important metric in stress testing. This
indicates the maximum number of users that can be served correctly, without errors
and hence application capacity.
Ideally error percentage should be zero throughout the test run. If error percent
is high, root cause of the error should be analyzed. It could be because of wrong
inputs to the load test, network connectivity problem or the server might have crashed.
Does the page response times satisfy target requirements?
Visual studio load testing has a builtin graph called "Page Response Time". This
response Time versus Elapsed Time report indicates the average response time of
the transactions over the elapsed time of the load test as shown.
From this graph it is possible to identify peaks in the response time over a period
of time. Ideal behavior of response time graph is that response time does not increase
with load. The point at which the graph increases sharply indicates beyond this
load server cannot serve the request and users will see no response or a very slow
response
Which one is the slowest page on web application during tests?
Same "Page Response Time" graph can be utilize for this purpose. This graph represents
average response time for the each page in time. Each bar in the graph is the average
server response time for each page.From this graph it possible to identify slowest
page of the application.
Which page take longest time to download?
Same "Page Response Time" graph can be utilize for this purpose too.Each point in
the graph is the average of the page download time during that interval. Page Download
time graph of individual pages help you to identify the exact page that has high
download time.
How many simultaneous users can web application handle?
Built in"Test Response Time" graph can be used for this purpose. If we analyze Response
Time Vs Test response Graph we can find out the capability of the web application.The
Response Time graph shows the time the server took for responding as the test load
is increased. The peak indicates that the server has reached its capacity to serve
data, and is unable to scale further.
Finally, visual studio load test has so much more to offer, this post only focused on those analysis that I thought important to get to know your application better. You can come up with lots of custom reports by utilize the result data if you need to. Hope this will help you.
References
Load Test Walk through
Run test in
Rig
Think profile/time
Add/Edit test database
History
V 1.0: TestingBox V 1.0 with config and analysis.
