Introduction
The main purpose of iCheckEngine is to aid testers in installer testing.
In general, during installer testing, a tester matches the test scenario provided in the Install Specification.
This includes test against:
- File(s) deployment - Test hundreds of files/folders if they are deployed, and for their file sizes and file content.
- Registry(s) - Test registry entries for their creation and value.
- Executable version - Test if the correct version of the executable is deployed.
- Database - Test pertaining to databases, i.e., Tables/Procedures/Triggers are created, and connectivity and password are correct.
- XML configuration - XML values in configuration are properly done.
- Settings for environment variables
Note: Above is an unexhausted list, there could be many more tests which need to done for an installer testing, inputs are welcome.
We felt the need for this utility when we had many installers to test. Therefore, I worked to make some part of it automatic.
iCheckEngine provides users with the components mentioned below to test scenarios related to installer testing.
At the end of each run, it can be configured to create either a file or HTML based report.
I was able to identify the following components which are required for a simple installation testing (inputs are most welcome).
- File - Files are properly deployed or not, their size, version, content, etc.
- Registry - Registry keys are created, values are set.
- XML - Many applications nowadays store their configuration in XML files, therefore checks related to XML nodes.
- Database - Checks pertaining to DB, tables, procedures, and triggers.
- String check - It is useful to test environment variables against some general string match.
Two kinds of report generation are supported - HTML and file based:
- FileLog - Provides functions useful for file based report generation.
- HtmlLog - Provides functions useful for HTML based report generation.
Understanding iCheckEngine
The iCheckEngine application can be divided into two parts:
- iCheckEngine server and parser: The iCheckEngine server and parser are integrated into the main iCheckEngine.exe. Components consist of a number of DLLs
which expose a
Check function to be called by iCheckEngine.exe. - iCheckEngine components: Components can be further divided into two parts:
- Internal - which are integrated into the main iCheckEngine.exe
- External - They expose check functions from the DLL
Note: External components need to be explicitly declared in the file iCheckComponentConfig.xml with tmethods they expose.
To keep things simple, these exported methods have a fixed signature as defined by the function pointer ISimpleFunc.
Given below are four important configuration files needed for iCheckEngine:
- iCheckConfig.xml - Main configuration files to be provided from command line arguments to iCheckEngine.
- Components file - Path for these files are mentioned in iCheckConfig.xml.
- Install check file - Path for these files is also mentioned in iCheckConfig.xml.
Most important: It is possible to mention multiple InstallCheck nodes.
All files will be processed and reports will be generated based on individual file configurations.
- Log file - File path application activity will be logged during parsing/execution of iCheckConfig.
Shown below is the sample content of iCheckConfig.xml:
<Install-Check version = "1.0.0.1">
<Components FilePath="C:\iCheckEngine\config\iCheckComponentConfig.xml" />
<LogFilePath FilePath="C:\iCheckEngine\Logs.txt" />
<InstallCheck FilePath="C:\iCheckEngine\config\iCheckSampleTest.xml" />
<InstallCheck FilePath="C:\iCheckEngine\config\iCheckSampleTest_2.xml" />
</Install-Check>
iCheckComponentConfig.xml contains the definition for external components and method they expose. To add new components, these files need to be modified appropriately.
Below is a sample content of the file iCheckComponentConfig.xml:
="1.0"="UTF-8"
<Install-Check>
<Components>
<File Enabled="TRUE" Path="C:\iCheckEngine\iFileComponent.dll">
<Methods>
<FileExistenceCheck />
<FileNotExistenceCheck />
<DirExistenceCheck />
<DirNotExistenceCheck />
<FileSizeGreater />
<FileSizeLesser />
<CheckFileVersion />
<MatchFileContent />
</Methods>
</File>
<Registry Enabled="TRUE" Path="C:\iCheckEngine\iRegistryComponent.dll">
<Methods>
<KeyExistenceCheck />
<KeyNotExistenceCheck />
<KeyValueCheck />
</Methods>
</Registry>
<FileLog Enabled="FALSE" Path="C:\iCheckEngine\iFileLog.dll">
<Methods>
<WriteReportHeader />
<WriteReportTrailer />
<WriteMessage />
</Methods>
</FileLog>
<HTMLLog Enabled="TRUE" Path="C:\iCheckEngine\iFileLog.dll">
<Methods>
<WriteHTMLReportHeader />
<WriteHTMLReportTrailer />
<WriteHTMLMessage />
</Methods>
</HTMLLog>
.....Content Stripped.................
Section in iCheckSampleTest.xml:
Variable - User defined variables (Note: Environment variables are automatically loaded).ExecuteAtStart - Component method to be called at start of check execution.ExecuteForEachCheck - Component method to be called at call of every check function.ExecuteAtEnd - Component method to be called at end of check execution.Aler<code>ts - Alerts need to be defined here.
iCheckSampleTest.xml holds actual instructions to call the appropriate install component functions for designated tasks.
Below is the sample content of a file which contains instructions for an installer check: iCheckSampleTest.xml:
<Install-Check>
<Variables>
<HTMLFilePath value ="C:\iCheckEngine\Reports\TestHtml.html" />
<ReportFilePath value ="C:\iCheckEngine\Reports\TestReport.txt" />
<ErrorFilePath value="C:\iCheckEngine\Reports\Data\Errors.txt" />
<HTMLHeaderTemplate value="C:\iCheckEngine\ReportTemplates\
HTMLHeaderTemplate.txt" />
<HTMLFootorTemplate value="C:\iCheckEngine\ReportTemplates\
HTMLFootorTemplate.txt" />
<HTMLMessageTemplate value="C:\iCheckEngine\ReportTemplates\
HTMLMessageTemplate.txt" />
<HeaderTemplate value=
"C:\iCheckEngine\ReportTemplates\HeaderTemplate.txt" />
<FootorTemplate value=
"C:\iCheckEngine\ReportTemplates\FootorTemplate.txt" />
<MessageTemplate value=
"C:\iCheckEngine\ReportTemplates\MessageTemplate.txt" />
</Variables>
<ExecuteAtStart>
<HTMLLog>
<WriteHTMLReportHeader>
<Parameter Filepath="@HTMLFilePath" HTMLHeaderTemplate="
@HTMLHeaderTemplate" HTMLFootorTemplate="@HTMLFootorTemplate"
HTMLMessageTemplate="@HTMLMessageTemplate"/>
</WriteHTMLReportHeader>
</HTMLLog>
<FileLog>
<WriteReportHeader>
<Parameter Filepath="@ReportFilePath" HeaderTemplate="
@HeaderTemplate" FootorTemplate="@FootorTemplate"
MessageTemplate="@MessageTemplate"/>
</WriteReportHeader>
</FileLog>
</ExecuteAtStart>
<ExecuteForEachCheck>
<FileLog>
<WriteMessage>
<Parameter Message="#OutputMessage" Status="#Status"
Check="#CheckName" Component="#Component" Method="#Method" />
</WriteMessage>
</FileLog>
<HTMLLog>
<WriteHTMLMessage>
<Parameter Message="#OutputMessage" Status="#Status"
Check="#CheckName" Component="#Component" Method="#Method" />
</WriteHTMLMessage>
</HTMLLog>
</ExecuteForEachCheck>
<Alerts>
<Check Id="Test">
<StringCheck>
<Equals>
<Parameter String1="@USERNAME" String2="
bd98101" CaseSensitive="true" />
<Parameter String1="@USERNAME" String2="
bd98101" CaseSensitive="false" />
</Equals>
<EqualsLike>
<Parameter String1="@OS" String2="windows"
CaseSensitive="true"/>
</EqualsLike>
</StringCheck>
<File>
<FileExistenceCheck>
<Parameter Filepath="C:\AUTOEXEC.BAT" />
<Parameter Filepath = "C:\FileNotExist.txt" />
</FileExistenceCheck>
<CheckFileVersion>
<Parameter Filepath = "C:\WINNT\TASKMAN.EXE"
Version = "5.0.2134.1"/>
</CheckFileVersion>
<MatchFileContent>
<Parameter File1 = "C:\iCheckEngine\TempFiles\File1.txt"
File2 = "C:\iCheckEngine\TempFiles\File2.txt"
CaseSensitive="true"/>
</MatchFileContent>
</File>
</Check>
<Check Id="CheckDirectoryExist">
<File>
<DirExistenceCheck>
<Parameter Dirpath="C:\Winnt"/>
</DirExistenceCheck>
<DirExistenceCheck>
<Parameter Dirpath="@windir\system32"/>
</DirExistenceCheck>
</File>
</Check>
<Check Id="CheckFileSizes">
<File>
<FileSizeGreater>
<Parameter Filepath="C:\Test.txt" size="10240" />
</FileSizeGreater>
<FileSizeLesser>
<Parameter Filepath="C:\Test.txt" size="10240" />
</FileSizeLesser>
</File>
</Check>
<Check Id="CheckRegistry">
<Registry>
<KeyExistenceCheck>
<Parameter KeyPath="SOFTWARE\Microsoft\
Internet Explorer\W2kVersion"
KeyNode="HKEY_LOCAL_MACHINE"/>
</KeyExistenceCheck>
<KeyValueCheck>
<Parameter ExpectedValue="6.0.2800.1106"
ValueName="W2kVersion"
KeyPath="SOFTWARE\Microsoft\Internet Explorer"
KeyNode="HKEY_LOCAL_MACHINE"/>
</KeyValueCheck>
</Registry>
</Check>
</Alerts>
<ExecuteAtEnd>
<HTMLLog>
<WriteHTMLReportTrailer>
<Parameter Filepath="@HTMLFilePath" GeneratedDateTime="
#CurrentDateTime" TotalCheck="#TotalCheck" SuccessCount="
#SuccessCount" FailedCount="#FailedCount"
iVersion = "#iVersion"/>
</WriteHTMLReportTrailer>
</HTMLLog>
<FileLog>
<WriteReportTrailer>
<Parameter Filepath="@ReportFilePath" GeneratedDateTime="
#CurrentDateTime" TotalCheck="#TotalCheck" SuccessCount="
#SuccessCount" FailedCount="#FailedCount" iVersion = "#iVersion"/>
</WriteReportTrailer>
</FileLog>
</ExecuteAtEnd>
</Install-Check>
I believe "A picture speaks 1000 words", and for coding, the output speaks for sabout 1000 lines of code...
Sample HTML Output
Sample Text Output
Thanks
Special thanks to Alex Hazanov for his XML Wrapper that helped in reading XML files.
History
- Updates 22-June-2009:
- Source download file:
- Fixed compiler issue pertaining to Debug and Release versions.
- Now uses generic paths to allow projects to be extracted in any folder and Run.
- Article text:
- Added HTML and TEXT output images to the article.
s
