Barcode detection and recognition technology, especially the application of real-time barcode scanning, is imperative to industries such as warehouse management and mobile payment. Dynamsoft Barcode Reader, as a cross-platform and cross-programing language SDK, is a lifesaver for developers who are dedicated to building enterprise-class barcode solutions in a short time.
In this post, we discuss how to leverage Dynamsoft Barcode Reader video decoding APIs to implement the barcode scanning functionality in camera preview scenario. Also, we demonstrate how to implement for desktop and mobile platforms respectively with the code snippet.
The Challenge of Camera Real-time Barcode Scanning
The process of reading barcodes consists of various computer vision algorithms, some of which are CPU-intensive. To make the camera preview run smoothly, you have to execute intensive computation asynchronously in a worker thread. A multithreading model is required for this case, in which developers have to spend a lot more time on thread scheduling, shared memory, status synchronization, and lock. Developers are prone to make mistakes in such a complicated scenario.
To eliminate the complexity of multi-threaded programming and improve developers’ efficiency, Dynamsoft Barcode Reader 7.0 brings a set of thread-based APIs for decoding continuous frames.
Dynamsoft Barcode Reader Overview
Dynamsoft's Barcode Reader SDK enables you to efficiently embed barcode reading functionality in your web, desktop or mobile application using just a few lines of code, which saves your development time and cost. With the SDK, you can create high-speed and reliable barcode scanner software to meet your business needs.
License
Get a FREE 30-day trial license.
Supported Barcode Symbologies
Linear Barcodes (1D): Code 39, Code 93, Code 128, Codabar, Interleaved 2 of 5, EAN-8, EAN-13, UPC-A, UPC-E, Industrial 2 of 5.
2D Barcodes: QR Code, DataMatrix, PDF417 and Aztec Code.
Developer’s Guide
https://www.dynamsoft.com/help/Barcode-Reader/devguide/index.html
API Documentation
https://www.dynamsoft.com/help/Barcode-Reader/index.html
Code Gallery
https://www.dynamsoft.com/Downloads/Dynamic-Barcode-Reader-Sample-Download.aspx
Online Demo
https://demo.dynamsoft.com/DBR/BarcodeReaderDemo.aspx
How Barcode Video Decoding APIs Work
The video decoding model contains a user main thread, a decoding thread, and a result consumer thread.
User Main Thread
The user main thread is initiated when StartFrameDecoding() is being called. Invoke AppendFrame() to add frames to an auto-managed frame queue in a video frame callback function.
Decoding Thread
The decoding thread takes all CPU-intensive work, running barcode algorithms. It uses some strategies to select and discard frames in order to avoid lagging.
Result Consumer Thread
This thread manages registered callback functions for fetching results.
Desktop Barcode Scanning
To build a desktop barcode app quickly, we recommend using Python and OpenCV.
Breaking Through Python’s GIL
The performance of Python multithreading code, especially for computation-intensive tasks, is restricted due to Python’s GIL (Global Interpreter Lock). The new video decoding APIs of Dynamsoft Barcode Reader is based on native C/C++ thread, which will not be affected by Python’s virtual machine.
Let’s take a glimpse of the Python sample code:
import cv2
import dbr
import time
import os
# The callback function for receiving barcode results
def onBarcodeResult(format, text):
print("Type: " + format)
print("Value: " + text + "\n")
def read_barcode():
video_width = 640
video_height = 480
vc = cv2.VideoCapture(0)
vc.set(3, video_width) #set width
vc.set(4, video_height) #set height
if vc.isOpened():
dbr.initLicense('LICENSE-KEY')
rval, frame = vc.read()
else:
return
windowName = "Barcode Reader"
max_buffer = 2
max_results = 10
barcodeTypes = 0x3FF | 0x2000000 | 0x4000000 | 0x8000000 | 0x10000000 # 1D, PDF417, QRCODE, DataMatrix, Aztec Code
image_format = 1 # 0: gray; 1: rgb888
dbr.startVideoMode(max_buffer, max_results, video_width, video_height, image_format, barcodeTypes, onBarcodeResult)
while True:
cv2.imshow(windowName, frame)
rval, frame = vc.read()
start = time.time()
try:
ret = dbr.appendVideoFrame(frame)
except:
pass
cost = (time.time() - start) * 1000
print('time cost: ' + str(cost) + ' ms')
# 'ESC' for quit
key = cv2.waitKey(1)
if key == 27:
break
dbr.stopVideoMode()
dbr.destroy()
cv2.destroyWindow(windowName)
if __name__ == "__main__":
print("OpenCV version: " + cv2.__version__)
read_barcode()
The dbr module is the Python extension built with Dynamsoft Barcode Reader. We created three methods: startVideoMode(), appendVideoFrame() and stopVideoMode(). The corresponding C/C++ code is as follows.
static PyObject *
startVideoMode(PyObject *self, PyObject *args)
{
printf("Start the video mode\n");
CHECK_DBR();
PyObject *callback = NULL;
int maxListLength, maxResultListLength, width, height, imageformat, iFormat, stride;
if (!PyArg_ParseTuple(args, "iiiiiiO", &maxListLength, &maxResultListLength, &width, &height, &imageformat, &iFormat, &callback)) {
return NULL;
}
updateFormat(iFormat);
if (!PyCallable_Check(callback))
{
PyErr_SetString(PyExc_TypeError, "parameter must be callable");
return NULL;
}
else
{
Py_XINCREF(callback); /* Add a reference to new callback */
Py_XDECREF(py_callback); /* Dispose of previous callback */
py_callback = callback;
}
ImagePixelFormat format = IPF_RGB_888;
if (imageformat == 0)
{
stride = width;
format = IPF_GRAYSCALED;
}
else {
stride = width * 3;
format = IPF_RGB_888;
}
DBR_SetTextResultCallback(hBarcode, onResultCallback, NULL);
int ret = DBR_StartFrameDecoding(hBarcode, maxListLength, maxResultListLength, width, height, stride, format, "");
return Py_BuildValue("i", ret);
}
appendVideoFrame()
static PyObject *
appendVideoFrame(PyObject *self, PyObject *args)
{
CHECK_DBR();
PyObject *o;
if (!PyArg_ParseTuple(args, "O", &o))
return NULL;
#if defined(IS_PY3K)
//Refer to numpy/core/src/multiarray/ctors.c
Py_buffer *view;
int nd;
PyObject *memoryview = PyMemoryView_FromObject(o);
if (memoryview == NULL) {
PyErr_Clear();
return -1;
}
view = PyMemoryView_GET_BUFFER(memoryview);
unsigned char *buffer = (unsigned char *)view->buf;
nd = view->ndim;
int len = view->len;
int stride = view->strides[0];
int width = view->strides[0] / view->strides[1];
int height = len / stride;
#else
PyObject *ao = PyObject_GetAttrString(o, "__array_struct__");
if ((ao == NULL) || !PyCObject_Check(ao)) {
PyErr_SetString(PyExc_TypeError, "object does not have array interface");
return NULL;
}
PyArrayInterface *pai = (PyArrayInterface*)PyCObject_AsVoidPtr(ao);
if (pai->two != 2) {
PyErr_SetString(PyExc_TypeError, "object does not have array interface");
Py_DECREF(ao);
return NULL;
}
// Get image information
unsigned char *buffer = (unsigned char *)pai->data; // The address of image data
int width = (int)pai->shape[1]; // image width
int height = (int)pai->shape[0]; // image height
int stride = (int)pai->strides[0]; // image stride
#endif
// Initialize Dynamsoft Barcode Reader
TextResultArray *paryResult = NULL;
// Detect barcodes
ImagePixelFormat format = IPF_RGB_888;
if (width == stride)
{
format = IPF_GRAYSCALED;
}
else if (width == stride * 3)
{
format = IPF_RGB_888;
}
else if (width == stride * 4)
{
format = IPF_ARGB_8888;
}
int frameId = DBR_AppendFrame(hBarcode, buffer);
return 0;
}
stopVideoMode()
static PyObject *
stopVideoMode(PyObject *self, PyObject *args)
{
printf("Stop the video mode\n");
if (hBarcode)
{
int ret = DBR_StopFrameDecoding(hBarcode);
return Py_BuildValue("i", ret);
}
return 0;
}
You have to call Python code between PyGILState_Ensure() and PyGILState_Release() when executing the callback function on the native thread:
void onResultCallback(int frameId, TextResultArray *pResults, void * pUser)
{
// Get barcode results
int count = pResults->resultsCount;
int i = 0;
// https://docs.python.org/2/c-api/init.html
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
for (; i < count; i++)
{
// https://docs.python.org/2.5/ext/callingPython.html
PyObject *result = PyObject_CallFunction(py_callback, "ss", pResults->results[i]->barcodeFormatString, pResults->results[i]->barcodeText);
if (result == NULL) return NULL;
Py_DECREF(result);
}
PyGILState_Release(gstate);
/////////////////////////////////////////////
// Release memory
DBR_FreeTextResults(&pResults);
}
Source Code
https://github.com/dynamsoft-dbr/python/tree/7.x
Android Barcode Scanning
To build a simple Android barcode scanning app with Android camera API and Dynamsoft Barcode Reader.
cameraView.addFrameProcessor(new FrameProcessor() {
@SuppressLint("NewApi")
@Override
public void process(@NonNull final Frame frame) {
try {
if (isDetected && isCameraOpen) {
YuvImage yuvImage = new YuvImage(frame.getData(), ImageFormat.NV21,
frame.getSize().getWidth(), frame.getSize().getHeight(), null);
if (width == 0) {
width = yuvImage.getWidth();
height = yuvImage.getHeight();
stride = yuvImage.getStrides()[0];
reader.setErrorCallback(errorCallback, null);
reader.setTextResultCallback(textResultCallback, null);
reader.setIntermediateResultCallback(intermediateResultCallback, null);
reader.startFrameDecoding(10, 10, width, height, stride, EnumImagePixelFormat.IPF_NV21, "");
} else {
PublicRuntimeSettings s = reader.getRuntimeSettings();
int frameid = reader.appendFrame(yuvImage.getYuvData());
Log.i("FrameId", frameid + "");
}
if (bUpateDrawBox) {
bUpateDrawBox = false;
Message message = handler.obtainMessage();
Rect imageRect = new Rect(0, 0, width, height);
message.obj = imageRect;
message.what = 0x001;
handler.sendMessage(message);
}
isDetected = true;
}
} catch (Exception e) {
e.printStackTrace();
}
}
});
Call startDecodingFrame() to initialize the environment and call appendFrame() repeatedly to add the preview images.
Source Code
https://github.com/dynamsoft-dbr/android-barcode-decode-video
iOS Barcode Scanning
To build a simple iOS barcode scanning app with iOS camera API and Dynamsoft Barcode Reader.
(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection;
{
isCurrentFrameDecodeFinished = NO;
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(imageBuffer, kCVPixelBufferLock_ReadOnly);
int bufferSize = (int)CVPixelBufferGetDataSize(imageBuffer);
void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);
CVPixelBufferUnlockBaseAddress(imageBuffer, kCVPixelBufferLock_ReadOnly);
NSData * buffer = [NSData dataWithBytes:baseAddress length:bufferSize];
startRecognitionDate = [NSDate date];
if (width != (int)CVPixelBufferGetWidth(imageBuffer)) {
width = (int)CVPixelBufferGetWidth(imageBuffer);
height = (int)CVPixelBufferGetHeight(imageBuffer);
stride = CVPixelBufferGetBytesPerRow(imageBuffer);
[m_barcodeReader setDBRErrorDelegate:self userData:nil];
[m_barcodeReader setDBRTextResultDelegate:self userData:nil];
[m_barcodeReader setDBRIntermediateResultDelegate:self userData:nil];
[m_barcodeReader startFrameDecoding:10 maxResultQueueLength:100 width:width height:height stride:stride format:EnumImagePixelFormatARGB_8888 templateName:@"" error:nil];
} else {
[m_barcodeReader appendFrame:buffer];
}
}
Call the following to initialize the environment and call -(NSInteger)appendFrame:(NSData*) bufferBytes repeatedly to add the preview images.
(void)startFrameDecoding:(NSInteger)maxQueueLength
maxResultQueueLength:(NSInteger)maxResultQueueLength
width:(NSInteger)width
height:(NSInteger)height
stride:(NSInteger)stride
format:(EnumImagePixelFormat)format
templateName:(NSString* _Nonnull)templateName
error:(NSError* _Nullable * _Nullable)error
Source Code
https://github.com/dynamsoft-dbr/ios-barcode-decode-video
Technical Support
If you have any questions about Dynamsoft Barcode Reader SDK, please feel free to contact support@dynamsoft.com.
Release History
v7.0, 07/12/2019
New
- Refactored most modules to provide a flexible barcode reading framework that allows for parameter customization suited for a variety of barcode scenarios
- Enabled access to intermediate results (grayscale image, binarized image, text zone, etc) during the decoding process
- Added new interfaces to support video decoding, and frame decoding to improve interactive sensitivity
- Provided methods to terminate the decoding process at different phases such as during binarization, localization or barcode type identification.
- Added a new barcode localization method, Scan Directly, to reduce decoding time significantly for high-quality images.
Improved
- Enhanced error messages related to license initiation failure.
- Improved detailed results for decoded barcodes, including more barcode format specification
- Improved results output to enable outputting barcode results in the order of confidence level, barcode position or format.
Fix
- Fixed an issue where the barcode could be calculated incorrectly on some occasions.
