Introduction
If you administer any Linux/UNIX servers using the VMSTAT Log file and want some tool or application which will help you to analyze the log file by displaying customizable graphs along with statistics for every component in every descriptor, then you are in the right place. Go ahead, read the article, download the application, and use it.
This application will demonstrate how to effectively use the VMSTAT Analyzer. The application is flexible enough for user configuration. The user needs to provide or locate the VMSTAT file which s/he wants to analyze for UNIX / Linux system resource utilization.
Below are listed some functionality which can be performed very easily using user friendly UI in the application:
- Once the user loads the VMSTAT Log file in the application, it checks for integrity and validity of the log file and prompts the user in case there are any warnings or errors. So, you sit back and relax, as VMSTAT_Analyzer.exe checks for the integrity and validity of the VMSTAT Log file.
- The user can view Server Resource Utilization for procs, memory, Swap, IO, System, CPU (as Field Descriptor), and also there are individual components in the customizable graph and other statistics data which is included in the VMSTAT Log file.
- This data is parsed from the VMSTAT file which the user has loaded in VMSTAT_Analyzer.exe and displays data dynamically.
- It also helps the user to know the meaning of each component for each Field Descriptor.
- It helps the user to know the Minimum, Average, Maximum, Percentile, and Standard Deviation value (as statistics) for each component for every Field Descriptor available in the log file.
- User has the option of Granularity / Zoom selection on the graph which extents to the graph broken down into small parts by dragging the mouse cursor inside the graph with a left click.
- The user can un-zoom or undo all zoom made by the user on the graph to full scale display by right clicking on the menu and selecting the option.
- X axis is calculated depending on the number of data points and converted into the HH:MM:SS format, which helps the user to understand how long the VMSTAT ran in the server.
- The application is flexible enough for user configuration.
- The user can change the Title, X Axis, and Y Axis for the graph as per choice, which overrides the default value.
- The user can change the interval for the data points which are available in the log file. This interval value entirely depends on the value that the user has mentioned in the VMSTAT command and is very important to configure because the X axis depends entirely on this interval value and the number of data points available in the VMSTAT log file. E.g., if the user used vmstat -n 1 > vmstat.log to collect VMSTAT Log file from the server, then set the interval time as 1. If the user uses vmstat -n 5 > vmstat.log, then set the interval as 5. This value is in seconds.
- The user can change the percentile value for the statistics. This value is in %.
- The user has the option of changing the line color for the graph which helps in greater visibility.
- The user can save the graph in various image formats in a desired location or in the Clipboard. This can be done very easily by right clicking on the graph and selecting the menu.
- The user can export statistics values in CSV format in a desired location. This can be done very easily by right clicking on the Statistics section and selecting the menu.
The most important note that everybody should consider when creating the VMSTAT file in their UNIX / Linux environment: Please use vmstat -n 1 > vmstat.log. You can redirect to any file name you want and any duration (interval for data point) for VMSTAT. But note the option -n. I am not going to describe anything about VMSTAT (which is out of the scope for this article), but remember that -n gets the desired header in the VMSTAT Log file by which VMSTAT_Analyzer.exe can understand and display customizable information.
Background
For a couple of days, I was searching in the Internet for something like the application of this article which will help me to show the graph for each component and also provide me their statistic values which should be configurable. But I couldn't find any.
That's when I decided to create one and distribute in www.codeproject.com, so that others can use it and get help out of the application.
Screenshot
Displaying customizable graph and statistics after you have loaded a valid VMSTAT log file which was created in the server using the command vmstat -n 1 > vmstat.log:
Displaying the user choice of selecting which Field Descriptor the user wants to view data.
Displaying the user choice of configuring the graph and various other factors:
The user can change the line color which effects the graph plotting for greater visibility:
Using the code
I have followed a very simple code and nothing is complicated down in the code. Anybody can get a grip of the code if you download the source file. You are free to make any modifications to the code. But it will be great if you can share it with others.
Let me brief about this portion of the code snippet. It's fairly simple. As you can see, the VMSTAT Log file has a specific header to categorize various server utilization matrix, which we get when we use the command vmstat -n 1 > vmstat.log in a UNIX / Linux server to create the log file. So the very first thing that VMSTAT_Analyzer.exe will do is to check for the integrity and validity of the VMSTAT log file which the user has loaded in the application. Below are some processes that this code snippet will perform.
- Count for number of data points available in the VMSTAT Log file to determine the value for the X Axis in the graph. It does [-2] to reduce the data point for the header string.
- It defines two variables for storing the header string which is constant when the user uses the command vmstat -n 1 > vmstat.log in the server.
- Then it reads each line from the VMSTAT Log file and breaks / splits that line delimited by space, so that each sub string can be read from each line.
- If the line read from the log file is the first line determined by the variable
intNumberOfDataPoint = 0, compare each sub string with the variable strFirstLineVMSTATArr. This will help to validate the first line in the VMSTAT Log. - Do the same for second line and compare with the variable
strSecondLineVMSTATArr, which will validate the second line in the VMSTAT Log. - From the third line till the end of the log file, it should contain the values for the individual component. So read each line and see that those values are integer values and each line should contain exactly 16 values.
If any of these above criteria is not matching, then throw an error and let the user know which line in the VMSTAT log file has an error and what is the error, so that user is aware of the error.
private void CheckIntegrity()
{
int intNumberOfDataPoint = 0;
intGlobalTotalRowsInVMSTATFile = 0;
string strVMSTATLineForTotalRows = null;
try
{
StreamReader readerTotalRows = File.OpenText(strGlobalVMSTATFile);
while ((strVMSTATLineForTotalRows = readerTotalRows.ReadLine()) != null)
{
intGlobalTotalRowsInVMSTATFile = intGlobalTotalRowsInVMSTATFile + 1;
}
readerTotalRows.Close();
stringArrayGlobalDataPointForX =
new string[intGlobalTotalRowsInVMSTATFile - 2];
StreamReader reader = File.OpenText(strGlobalVMSTATFile);
string[] strFirstLineVMSTATArr = { "procs",
"-----------memory----------",
"---swap--", "-----io----",
"--system--", "----cpu----" };
string[] strSecondLineVMSTATArr = { "r", "b", "swpd",
"free", "buff", "cache", "si",
"so", "bi", "bo", "in", "cs",
"us", "sy", "id", "wa" };
string strVMSTATEachLine = null;
string[] strVMSTATEachLineArray;
bool boolLineFlag = false;
ArrayList arrayListVMSTATLine = new ArrayList();
while ((strVMSTATEachLine = reader.ReadLine()) != null)
{
strVMSTATEachLineArray = strVMSTATEachLine.Split(' ');
if (intNumberOfDataPoint == 0)
{
foreach (string strFirstLineVMSTAT in strFirstLineVMSTATArr)
{
boolLineFlag = false;
foreach (string strParseLine in strVMSTATEachLineArray)
{
if (strParseLine.CompareTo(strFirstLineVMSTAT) == 0)
{
boolLineFlag = true;
break;
}
}
if (boolLineFlag == false)
{
if (MessageBox.Show("\"" + strFirstLineVMSTAT +
"\" is not avalibale. The file may be invalid " +
"or corrupt.\nThere may be error in parsing the file. Do you " +
"want to continue ?", "VMSTAT Error",
MessageBoxButtons.YesNo,
MessageBoxIcon.Question) == DialogResult.No)
{
Application.Exit();
throw new Exception("User has terminated the process.");
}
}
}
}
if (intNumberOfDataPoint == 1)
{
foreach (string strSecondLineVMSTAT in strSecondLineVMSTATArr)
{
boolLineFlag = false;
foreach (string strParseLine in strVMSTATEachLineArray)
{
if (strParseLine.CompareTo(strSecondLineVMSTAT) == 0)
{
boolLineFlag = true;
break;
}
}
if (boolLineFlag == false)
{
if (MessageBox.Show("\"" + strSecondLineVMSTAT +
"\" is not avalibale. The file may be invalid or " +
"corrupt.\nThere may be error" +
" in parsing the file. Do you want " +
"to continue ?", "VMSTAT Error",
MessageBoxButtons.YesNo,
MessageBoxIcon.Question) == DialogResult.No)
{
Application.Exit();
throw new Exception("User has terminated the process.");
}
}
}
}
if (intNumberOfDataPoint >= 2)
{
boolLineFlag = false;
arrayListVMSTATLine.Clear();
foreach (string strParseLine in strVMSTATEachLineArray)
{
if (strParseLine.CompareTo("") != 0)
{
int intParseLine = int.Parse(strParseLine);
arrayListVMSTATLine.Add(intParseLine);
}
}
object[] objStringArr = arrayListVMSTATLine.ToArray();
if (objStringArr.Length == 16)
{
boolLineFlag = true;
}
if (boolLineFlag == false)
{
if (MessageBox.Show("Line number : " + intNumberOfDataPoint +
" is not valid. The file may " +
"be invalid or corrupt or some value " +
"is missing in mentioned line " +
"number.\nThere may be error in parsing the file. " +
"Do you want to continue ?", "VMSTAT Error",
MessageBoxButtons.YesNo,
MessageBoxIcon.Question) == DialogResult.No)
{
Application.Exit();
throw new Exception("User has terminated the process.");
}
}
}
intNumberOfDataPoint = intNumberOfDataPoint + 1;
}
reader.Close();
}
catch (Exception ee)
{
MessageBox.Show(ee.Message + "\n\nLine number : " +
intNumberOfDataPoint + " is not valid. " +
"The file may be invalid or corrupt " +
"or some value is missing in mentioned line number.",
"VMSTAT Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
This is one of the major code snippets used to draw the graph by plotting the X and Y axes values along with other information, like description and graph title.
This method determines which Field Descriptor to consider and displays the graph for that Field Descriptor which the user has selected as per their needs. This is determined by the global variable intGlobalGraphFieldDescriptorValue.
private void CreateGraph(ZedGraphControl zgc)
{
GraphPane myPane = zgc.GraphPane;
myPane.Title.Text = strGlobalGraphTitle;
myPane.XAxis.Title.Text = strGlobalGraphXAxis;
myPane.YAxis.Title.Text = strGlobalGraphYAxis;
if (intGlobalGraphFieldDescriptorValue == 0)
{
ClearValues(zedGraphControl);
lblSubDesc1.Text = "r: The number of processes waiting for run time.";
lblSubDesc2.Text = "b: The number of processes in uninterruptible sleep.";
LineItem myCurve1 = myPane.AddCurve("r", null, CreatePointForY(0),
toolStripLine1.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve2 = myPane.AddCurve("b", null,
CreatePointForY(1), toolStripLine2.ForeColor,
SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
}
if (intGlobalGraphFieldDescriptorValue == 1)
{
ClearValues(zedGraphControl);
lblSubDesc1.Text = "swpd: the amount of virtual memory used.";
lblSubDesc2.Text = "free: the amount of idle memory.";
lblSubDesc3.Text = "buff: the amount of memory used as buffers.";
lblSubDesc4.Text = "cache: the amount of memory used as cache.";
LineItem myCurve1 = myPane.AddCurve("swpd", null,
CreatePointForY(2), toolStripLine1.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve2 = myPane.AddCurve("free", null,
CreatePointForY(3), toolStripLine2.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve3 = myPane.AddCurve("buff", null,
CreatePointForY(4), toolStripLine3.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve4 = myPane.AddCurve("cache", null,
CreatePointForY(5), toolStripLine4.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
}
if (intGlobalGraphFieldDescriptorValue == 2)
{
ClearValues(zedGraphControl);
lblSubDesc1.Text = "si: Amount of memory swapped in from disk (/s).";
lblSubDesc2.Text = "so: Amount of memory swapped to disk (/s).";
LineItem myCurve1 = myPane.AddCurve("si", null,
CreatePointForY(6), toolStripLine1.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve2 = myPane.AddCurve("so", null,
CreatePointForY(7), toolStripLine2.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
}
if (intGlobalGraphFieldDescriptorValue == 3)
{
ClearValues(zedGraphControl);
lblSubDesc1.Text = "bi: Blocks received from a block device (blocks/s).";
lblSubDesc2.Text = "bo: Blocks sent to a block device (blocks/s).";
LineItem myCurve1 = myPane.AddCurve("bi", null,
CreatePointForY(8), toolStripLine1.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve2 = myPane.AddCurve("bo", null,
CreatePointForY(9), toolStripLine2.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
}
if (intGlobalGraphFieldDescriptorValue == 4)
{
ClearValues(zedGraphControl);
lblSubDesc1.Text = "in: The number of interrupts per second, including the clock.";
lblSubDesc2.Text = "cs: The number of context switches per second.";
LineItem myCurve1 = myPane.AddCurve("in", null,
CreatePointForY(10), toolStripLine1.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve2 = myPane.AddCurve("cs", null,
CreatePointForY(11), toolStripLine2.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
}
if (intGlobalGraphFieldDescriptorValue == 5)
{
ClearValues(zedGraphControl);
lblSubDesc1.Text = "us: Time spent running non-kernel code. " +
"(user time, including nice time)";
lblSubDesc2.Text = "sy: Time spent running kernel code. (system time)";
lblSubDesc3.Text = "id: Time spent idle.";
lblSubDesc4.Text = "wa: Time spent waiting for IO.";
lblSubDesc5.Text = "cu: Total CPU Usage.";
LineItem myCurve1 = myPane.AddCurve("us", null,
CreatePointForY(12), toolStripLine1.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve2 = myPane.AddCurve("sy", null,
CreatePointForY(13), toolStripLine2.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve3 = myPane.AddCurve("id", null,
CreatePointForY(14), toolStripLine3.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve4 = myPane.AddCurve("wa", null,
CreatePointForY(15), toolStripLine4.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
LineItem myCurve5 = myPane.AddCurve("cu", null,
CreatePointForY(16), toolStripLine5.ForeColor, SymbolType.None);
myPane.XAxis.Scale.TextLabels = stringArrayGlobalDataPointForX;
}
zgc.AxisChange();
RepaintScreen();
}
After the graph is displayed by plotting the X and Y axes and other information like statistics value, checkboxes will be enabled as per the selected Field Descriptor. E.g., if the user selected PROCS as the Field Descriptor, then two checkboxes will be enabled for run time process "r" and uninterruptible sleep process "b". That means the graph will display two lines separated by different colors.
The user can filter by selecting one checkbox and unselecting another checkbox of any component, thus enabling them to analyze one component at a time.
private void FilteredCreateGraph(ZedGraphControl zgc)
{
GraphPane myPane = zgc.GraphPane;
if (intGlobalGraphFieldDescriptorValue == 0)
{
if (checkBoxViewGraph1.Checked == true)
{
LineItem myCurve1 = myPane.AddCurve("r", null,
CreatePointForY(0), toolStripLine1.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph2.Checked == true)
{
LineItem myCurve2 = myPane.AddCurve("b", null,
CreatePointForY(1), toolStripLine2.ForeColor, SymbolType.None);
}
}
if (intGlobalGraphFieldDescriptorValue == 1)
{
if (checkBoxViewGraph1.Checked == true)
{
LineItem myCurve1 = myPane.AddCurve("swpd", null,
CreatePointForY(2), toolStripLine1.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph2.Checked == true)
{
LineItem myCurve2 = myPane.AddCurve("free", null,
CreatePointForY(3), toolStripLine2.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph3.Checked == true)
{
LineItem myCurve3 = myPane.AddCurve("buff", null,
CreatePointForY(4), toolStripLine3.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph4.Checked == true)
{
LineItem myCurve4 = myPane.AddCurve("cache", null,
CreatePointForY(5), toolStripLine4.ForeColor, SymbolType.None);
}
}
if (intGlobalGraphFieldDescriptorValue == 2)
{
if (checkBoxViewGraph1.Checked == true)
{
LineItem myCurve1 = myPane.AddCurve("si", null,
CreatePointForY(6), toolStripLine1.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph2.Checked == true)
{
LineItem myCurve2 = myPane.AddCurve("so", null,
CreatePointForY(7), toolStripLine2.ForeColor, SymbolType.None);
}
}
if (intGlobalGraphFieldDescriptorValue == 3)
{
if (checkBoxViewGraph1.Checked == true)
{
LineItem myCurve1 = myPane.AddCurve("bi", null,
CreatePointForY(8), toolStripLine1.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph2.Checked == true)
{
LineItem myCurve2 = myPane.AddCurve("bo", null,
CreatePointForY(9), toolStripLine2.ForeColor, SymbolType.None);
}
}
if (intGlobalGraphFieldDescriptorValue == 4)
{
if (checkBoxViewGraph1.Checked == true)
{
LineItem myCurve1 = myPane.AddCurve("in", null,
CreatePointForY(10), toolStripLine1.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph2.Checked == true)
{
LineItem myCurve2 = myPane.AddCurve("cs", null,
CreatePointForY(11), toolStripLine2.ForeColor, SymbolType.None);
}
}
if (intGlobalGraphFieldDescriptorValue == 5)
{
if (checkBoxViewGraph1.Checked == true)
{
LineItem myCurve1 = myPane.AddCurve("us", null,
CreatePointForY(12), toolStripLine1.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph2.Checked == true)
{
LineItem myCurve2 = myPane.AddCurve("sy", null,
CreatePointForY(13), toolStripLine2.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph3.Checked == true)
{
LineItem myCurve3 = myPane.AddCurve("id", null,
CreatePointForY(14), toolStripLine3.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph4.Checked == true)
{
LineItem myCurve4 = myPane.AddCurve("wa", null,
CreatePointForY(15), toolStripLine4.ForeColor, SymbolType.None);
}
if (checkBoxViewGraph5.Checked == true)
{
LineItem myCurve5 = myPane.AddCurve("cu", null,
CreatePointForY(16), toolStripLine5.ForeColor, SymbolType.None);
}
}
zgc.AxisChange();
RepaintScreen();
}
There are various other methods down in the code which you may want to explore:
UncheckAllToolStrip(object sender) - This will help to uncheck all ToolStripMenuItems for the Field Descriptor.SetSizeForZedGraphControl() - This helps to set the size of the Graph or Charting control.ClearValues(ZedGraphControl zgc) - This helps to clear all the values in the Windows form.ClearGraph(ZedGraphControl zgc) - It only clear the values inside Graph.CreatePointForX() - This helps to create the values for X Axis in the format HH:MM:SS.CreatePointForY(int intGraphField) - This helps to create the values for the Y Axis.DisplayMinAvgMaxPercStdDValue(int intMin, decimal decimalAvg, int intMax, double doublePercentileValue, double doublePopStdDeviation, int intGraphField) - This helps to display the statistics value in the Windows form depending on the user selected Field Descriptor.GetPercentileValue(double[] intArrayYValue) - This helps to calculate the percentile value for an individual component.GetPopulationStdDeviationValue(double[] intArrayYValue) - This helps to calculate the Standard Deviation value for an individual component.Export_To_CSV(string strFileName) - This helps to export the statistics value in the user desired location in CSV format.
If you want to make some changes in the code, then add a reference to ZedGraph.dll (which is provided in the source code, or download from zedgraph.org, and is the basic Charting control) by selecting Project -> Add Reference.
Sample VMSTAT Log file
//VMSTAT Log file looks like below data
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 224 7016764 543576 22527872 0 0 4 16 0 0 4 1 96 0
0 0 224 7016700 543576 22527872 0 0 0 0 320 1317 0 0 99 0
0 0 224 7014812 543576 22527872 0 0 0 200 284 1171 1 0 98 0
0 0 224 7016508 543576 22527872 0 0 0 168 235 1173 0 0 100 0
...
...
If you want to download the sample VMSTAT Log file (which I used in this example), then here it is:
Points of interest
The user gets a beautiful chart for every component in the VMSTAT file along with their statistics which s/he can export as a graph in various image formats and the statistics in CSV format to a location of the user's choice.
Thanks
I like to thank zedgraph.org for their excellent charting class library. Kudos to zedgraph who made my life easy.
History and future enhancements
This is the first version of VMSTAT_Analyzer. I am planning to come up with another version for the -a option of VMSTAT which provides active memory distribution.
Please let me know if I can add any other functionality to the existing version which can help VMSTAT_Analyzer to be a better application.
