Data collected on paper forms ultimately needs to be stored in a system to be useful. A common practice that businesses take is to hire data entry clerks to manually input data from the paper forms into their systems one by one. This is a very time consuming, error-prone, and expensive process. A better solution for capturing data from paper-based sources is digital automation.
In this article, we will walk you through a streamlined process that covers the creation, printing, scanning, and automating of the data import of a paper-based form using the Aspose.OMR API.
What is OMR?
OMR stands for Optical Mark Recognition, commonly used on paper forms representing answers to multiple choice questions. People using a pen or pencil fill in a bubble to signify their answer to a question. OMR is the technology used to automate the retrieval of the information recorded on these forms through the use of imaging technology such as a scan or a photograph.
One thing to keep in mind when using OMR technology: OMR is not OCR (optical character recognition) — its use is strictly to detect answers marked on a sheet in the form of a filled bubble. It is, however, possible to combine the two technologies in a single software program.
Aspose.OMR is a .NET Framework-based OMR API meant for on-premise software. This API is compatible with all .NET Framework projects such as Windows Forms, WPF, ASP.NET, and WCF using the C# or VB.Net languages.
Aspose.OMR features form image creation, form template generation, form processing from scans or photos, and the export of extracted results into CSV, XML, or JSON formats. Aspose.OMR also has multiple cloud based apis available for those that desire to off-load processing to a third-party.
Implementing the OMR Workflow using the Aspose.OMR API
A complete OMR workflow can be achieved in as little as seven steps. We will walk through these steps and implement the automation of a full OMR workflow using a simple C# .NET Framework console application and the Aspose.OMR API.
Start by opening Visual Studio 2019 (Community version is adequate) and creating a new C# .NET Framework Console application.
Name the application "AsposeOMRWorkflow" and place it in the file location of your choice. Press the "Create" button to generate the project files.
Now that the project is loaded, we are ready to bring in the Aspose.OMR Api through the NuGet Package Manager. Right-click on the project file, and select the "Manage NuGet Packages" item.
In the NuGet Package manager, press the "Browse" tab and search for "Aspose.OMR", select it from the list, and press the install button to add it to the project.
Since we will be automating the scanning of filled out paper forms in this project, we will require an additional reference in our project. Right-click the project file once more and select "Add Reference".
In the add reference dialog, select "COM" from the left-hand menu, and search for "Windows Image". This will bring up a library for the Microsoft Windows Image Acquisition Library v2.0. Select this library and press the "OK" button to add it to the project. The short form for this library is referred to as WIA within our source code.
Step 1 - Define the form
The Aspose.OMR API uses a proprietary textual markup when defining the contents of a form. Form items may also have optional attributes. These attributes are always defined on a new line underneath the parent item, indented with a tab space. You will see examples of this below. Forms can be made up of many types of items as follows:
Regular Text
Markup
?text=<textual content goes here>
Use regular text when you wish to output textual items on the form that are not to be interpreted by the OMR engine. This could be things like a name and date line, a signature line, or general instructions for the form. Spacing in the text is preserved on the generated form.
Questions
Markup
#What is your question?
() Answer A () Answer B
() Answer C () Answer D
Questions are output using default numbering and default bubble values, which are letters in alphabetical order. To use the default value in the bubbles simply indicate a bubble with the left and right round parenthesis. You may put bubbles and answers on multiple lines, and the form will render accordingly.
Questions with customized answer values
Markup
#How do you rate this question? 5 for good, 1 for bad.
(5) (4) (3) (2) (1)
When the OMR engine template processor runs, it will return the value specified between the left and right round parentheses. You are not limited to default letter answers. You can use any in the bubble as defined by the markup above. If you try to utilize more than one character, the form will render with the first character in the parenthesis.
Grid
Markup
?grid=<Name of the field>
sections_count=<number of bubbles per line>
A grid is a vertical form item. Use a grid when inputting large numeric values such as student and membership IDs. The sections_count attribute allows you to define how many numbers are representative of the final value. The bubbles for each number, 0 through 9, are rendered beneath a line representing a numeric character in the value.
Answer Sheet
Markup
?answer_sheet=<name of section>
elements_count=<number of questions>
columns_count=<number of columns when outputting questions>
Many tests have a secondary text or booklet that contains the question text. In this case, the questions do not need to be repeated on the form. Instead, only the question numbers and bubbles need to be rendered. The answer sheet item allows you to display a specified number of questions (elements_count attribute) using a specified number of columns (columns_count attribute).
Let's go ahead and define our custom questionnaire that we will use in our console application. Add a new text file to the project and name it "QuestionnaireMarkup.txt", populate it with the markup specified below:
?text=Name__________________________________ Date____________
?grid=ID
sections_count=5
#What are some of the functions of Aspose.OMR? Mark all that apply.
() OCR () Capture human-marked data
() Generate data collection forms () Enhance images
#What is your favorite color?
(r) Red (g) Green
(b) Blue (o) Other
#Rate this question (5 - good, 1 - bad)
(5) (4) (3) (2) (1)
#Are you currently in a good mood?
(Yes) Yes (No) No
?text= Booklet #1 Answer Sheet
?answer_sheet=Booklet1
elements_count=15
columns_count=3
?text=Sign__________________________________
In the properties of the QuestionnaireMarkup.txt (right-click file, select "Properties"), ensure the "Copy to Output Directory" is set to "Copy if newer".
Step 2 - Generate the template and form image file
It's now time to use the OMR engine to generate a template and form image file based on the markup we defined for our questionnaire. The template is generated as a JSON-based file (with the extension .omr). This file is used internally by the template processor to describe the location of each of the elements on the form.
The form image is a PNG image that will be printed on paper for people to fill out. This file has boxes in the corners that act as registration points for when the template processor applies the location data from the generated template to interpret the information that has been filled out in the form.
To generate a template and form image, add a method to the Program class as follows (code documented inline):
//you will need to add the following using statement at the top of the file:
//using Aspose.OMR.Api;
static void GenerateFormTemplateAndImage()
{
//fully qualified path to the form markup file
var formMarkupFilePath = AppDomain.CurrentDomain.BaseDirectory +
"QuestionnaireMarkup.txt";
//initialize an instance of the Aspose OMR engine
var omrEngine = new OmrEngine();
//use the Aspose OMR engine to generate the template and
//image from the markup file
var result = omrEngine.GenerateTemplate(formMarkupFilePath);
if (result.ErrorCode != 0)
{
Console.WriteLine($"ERROR: {result.ErrorCode} -
{result.ErrorMessage}");
}
else
{
//save the files as OmrOutput.omr for the template,
//and OmrOutput.png for the form image
result.Save("", "OmrOutput");
}
}
As specified in the code, this method will generate two files based on the markup file we created in Step 1. The .omr file contains the location information of elements of the form in JSON format, and the .png file will contain the image of the form that will be used to print on paper and distributed to the people that will fill it out with pen or pencil.
Step 3 - Print the form image
The next step is to get the form image onto paper. To do this, we will add another method to the Program class (code documented inline):
//this method requires the following using statements at the top of the file
// using System.Drawing;
// using System.Drawing.Printing;
static void PrintFormImage()
{
//fully qualified path to the form image
var questionnaireImagePath = AppDomain.CurrentDomain.BaseDirectory +
"OmrOutput.png";
//we will be using the default printer to print the form
PrintDocument pd = new PrintDocument();
//event handler fired when a print job is requested
pd.PrintPage += (sender, args2) =>
{
//obtain image to print
Image i = Image.FromFile(questionnaireImagePath);
//obtain paper margins
Rectangle m = args2.MarginBounds;
//ensure we scale the image proportionally on the page
if ((double)i.Width / (double)i.Height >
(double)m.Width / (double)m.Height) // image is wider
{
m.Height = (int)((double)i.Height / (double)i.Width *
(double)m.Width);
}
else
{
m.Width = (int)((double)i.Width / (double)i.Height *
(double)m.Height);
}
//render the image
args2.Graphics.DrawImage(i, m);
};
//prints one copy of the form image
pd.Print();
}
Steps 4 and 5 - Fill out the form and Obtain a digital image of the filled form
Step 4 is the only step that requires manual intervention. Distribute the form and have people fill it out using a pen or pencil, then collect the filled out forms.
With the filled out forms collected, we proceed to Step 5 — digitally imaging the forms. This can be done in a number of ways, such as taking a photograph with a digital camera, using a flatbed scanner, or using an ADF (Automatic Document Feeder) scanner.
In this example, the code is implemented to use an ADF scanner, but may be modified to use a flatbed as well. If using a digital camera, this step may be skipped — it is important however to have the paper form take up the majority of the picture, so cropping may be necessary.
Add the following method to scan multiple documents from an ADF scanner (documented inline):
//this method requires the following using statements at the top of the file
// using System.Linq;
// using System.Collections.Generic;
// using System.IO;
// using System.Drawing.Imaging;
static void ScanForms(int numberOfForms)
{
//fully qualified output path where the scanned images of the
//forms will be located.
var scannedImagePath = AppDomain.CurrentDomain.BaseDirectory + @"Scans\";
//obtain the first defined scanner found on the computer
var scanner = WIAScanner.GetDevices().FirstOrDefault();
List<Image> images = new List<Image>();
int idx = 0;
//scans multiple forms and stores the images in the images list
images = WIAScanner.Scan(scanner.DeviceID, numberOfForms,
WIAScanQuality.Final, WIAPageSize.Letter, DocumentSource.Feeder);
if (images.Count > 0)
{
//create the Scans directory if it doesn't already exist
Directory.CreateDirectory(scannedImagePath);
}
//save each image obtained from the scanner into the Scans folder.
foreach (var img in images)
{
var fileName = "img" + ++idx;
img.Save(scannedImagePath + fileName + ".jpg", ImageFormat.Jpeg);
}
}
Note: In the source code provided in the Github repository, a helper class (WIAScanner.cs) has been added to take care of much of the plumbing code necessary to control a scanning device. We will not be stepping through this code as it is not the focus of this article. Obtain this file from the repository and add it to your project.
Steps 6 and 7 - Process the digital image(s) and Export the collected data
Now that we have a collection of images scanned and stored on the file system, we can use the OMR engine template processor to extract the results of our questionnaire. The template processor uses the registration marks (the squares on the form) to identify the location of the form elements based on the template that we generated in step 2 (the .omr file).
Implement the following method in the Program class to loop through the images in the Scans directory, process the information, and export it to JSON files. Currently we are storing the JSON information on the file system, but it could also be delivered to the location of your choosing, such as in a database or sent to a web service.
static void ProcessScansAndExportData()
{
//fully qualified output path where the scanned images
//of the forms will be located.
var scannedImagePath = AppDomain.CurrentDomain.BaseDirectory +
@"Scans\";
///fully qualified path to the form template
var templatePath = AppDomain.CurrentDomain.BaseDirectory +
"OmrOutput.omr";
//initialize an instance of the Aspose OMR engine
var omrEngine = new OmrEngine();
//retrieve all images from the Scans folder
var dirInfo = new DirectoryInfo(scannedImagePath);
var files = dirInfo.GetFiles("*.jpg");
//use the omrEngine to create an instance of the template
//processor based on the generated template
var templateProcessor2 = omrEngine.GetTemplateProcessor(templatePath);
foreach (var file in files)
{
//use the template processor to extract form data from the form image
string jsonResults =
templateProcessor2.RecognizeImage(file.FullName, 28).GetJson();
//save the extracted data in a json file
File.WriteAllText(AppDomain.CurrentDomain.BaseDirectory +
Path.GetFileNameWithoutExtension(file.FullName) +
"_scan_results.json", jsonResults);
}
}
Take a look at the line of code where the template processor is evaluating a form image using the RecognizeImage method. This method takes up to two parameters: the digital image of the form and the threshold value of the template engine.
The threshold value can be modified to indicate how sensitive the evaluation process is. This value can range from 0 to 100, with a higher value representing how strict the method will interpret the way the bubble is filled in on the form.
Depending on your situation, you may wish to modify this value to obtain optimal performance for your requirements. One way to visually determine a threshold value is to use the Open Source GUI tool provided by Aspose to load your template and sample form image and use the graphical control to determine a suitable value.
Setting up the OMR workflow automation
We will complete the implementation of our console application by outputting a menu system that will exercise the methods that we've already created. Replace the Main method in Program.cs with the following code:
static void Main(string[] args)
{
License omrLicense = new License();
omrLicense.SetLicense("Aspose.Total.lic");
var menuSelection = "";
while (menuSelection != "X" && menuSelection != "x")
{
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine("Generate Template - T");
Console.WriteLine("Print Questionnaire - P");
Console.WriteLine("Scan Flatbed Docs - S");
Console.WriteLine("Evaluate Scanned Docs - E");
Console.WriteLine("Exit - X");
Console.ForegroundColor = ConsoleColor.White;
menuSelection = Console.ReadLine();
switch (menuSelection)
{
case "T":
case "t":
GenerateFormTemplateAndImage();
break;
case "P":
case "p":
PrintFormImage();
break;
case "S":
case "s":
ScanForms(4);
break;
case "E":
case "e":
ProcessScansAndExportData();
break;
case "X":
case "x":
default:
break;
}
Console.BackgroundColor = ConsoleColor.DarkGreen;
Console.WriteLine("Finished");
Console.BackgroundColor = ConsoleColor.Black;
}
}
When the application is run, a menu is displayed allowing the user to automate each step of the OMR workflow. They may exit the application at any time by entering the letter "X".
Interpreting the results
Here is an example of a scanned form and its associated results in JSON format:
Results:
{
"RecognitionResults": [{
"ElementName": "Question1",
"Value": "B,C"
}, {
"ElementName": "Question2",
"Value": "b"
}, {
"ElementName": "Question3",
"Value": "3"
}, {
"ElementName": "Question4",
"Value": "Y"
}, {
"ElementName": "Booklet15",
"Value": "A"
}, {
"ElementName": "Booklet110",
"Value": "A"
}, {
"ElementName": "Booklet115",
"Value": "B"
}, {
"ElementName": "Booklet16",
"Value": "C"
}, {
"ElementName": "Booklet111",
"Value": "A"
}, {
"ElementName": "Booklet116",
"Value": "A"
}, {
"ElementName": "Booklet17",
"Value": "B"
}, {
"ElementName": "Booklet112",
"Value": "B"
}, {
"ElementName": "Booklet117",
"Value": "D"
}, {
"ElementName": "Booklet18",
"Value": "D"
}, {
"ElementName": "Booklet113",
"Value": "C"
}, {
"ElementName": "Booklet118",
"Value": "B"
}, {
"ElementName": "Booklet19",
"Value": "D"
}, {
"ElementName": "Booklet114",
"Value": "D"
}, {
"ElementName": "Booklet119",
"Value": "C"
}, {
"ElementName": "ID",
"Value": "12345"
}]
}
The template processor achieved 100% accuracy with this form. You can see Question1 through Question4 matching the first 4 questions on the form. You can identify our Booklet 1 answer sheet answers through the element name having the prefix of "Booklet1", meaning "Booklet15" is question number 5 of Booklet 1. Finally, our grid, which we identified in our template as "ID" is available as the last item in the array combines the values filled in on the form as a single value.
Conclusion
Using the Aspose.OMR API library in .Net-based projects greatly simplifies the OMR process. In this article, we implemented a sample OMR workflow from end-to-end.
A highlight of this library is the flexibility of defining thresholds to allow for fine-tuning results with the ultimate precision based on your scenario and form (for instance, a form filled out by elementary students vs. adults).
From a developer perspective, the inclusion of the library via NuGet and the programmatic loading of the license file is painless. The API itself is also clear and concise, making the developer experience a positive one.
Github
Github Aspose
