Introduction
There are several ways to grab and process webcamera images: WIA, DirectShow, VFW... There are lots of C# VFW examples on the Internet and most of them use .NET clipboard to transfer each frame's data from buffer to Bitmap
-recognizable object. Unfortunately, this makes multithreading unavailable and reduces FPS (frames per second). The native Win32 clipboard and multithreading solve the speed problem, but I thought that it wasn't the most elegant solution and there should be another way to get frames from Avicap. I have referred to MSDN (see VFW link above) and discovered that function callback was available. This article explains, step-by-step, how to capture frames using avicap32.dll (VFW) in a multi-thread environment.
The Idea
This is an approach that you can find in lots of examples from the Web:
- Create a capture window.
- Connect the capture window to the device.
- Set the video format (height and width in pixels).
- Capture the frame to temporary unreachable buffer.
- Copy the contents of the video frame buffer and associated palette to the clipboard.
- Get the frame from the clipboard, converting data to RGB
Bitmap . - Process
Bitmap . Go to Step 4.
| This is an approach I worked with:
- Create a capture window.
- Connect the capture window to the device.
- Set the video format (height and width in pixels, bits per frame).
- Capture the frame to temporary unreachable buffer.
- Make a callback (make buffer data available in
VIDEOHDR structure). - Get the frame data, converting data to RGB
Bitmap . - Process
Bitmap . Go to Step 4.
|
API
[DllImport("avicap32.dll", EntryPoint = "capCreateCaptureWindow")]
static extern int capCreateCaptureWindow(string lpszWindowName,
int dwStyle, int X, int Y,
int nWidth, int nHeight, int hwndParent, int nID);
The capCreateCaptureWindow function creates a new window for video stream capturing and returns its handle. This function is called in step 1.
[DllImport("user32", EntryPoint = "SendMessage")]
static extern bool SendMessage(int hWnd, uint wMsg, int wParam, int lParam);
The SendMessage function sends the specified message to a window or windows. It calls the window procedure for the specified window and does not return until the window procedure has processed the message. Note that both wParam
and lParam
specify additional message-specific information, i.e. numbers, pointers to structures, buffers. SendMessage
is overloaded in this project with different lParam
types. It is called in steps 2, 3, 4 and 5.
[DllImport("avicap32.dll")]
static extern bool capGetDriverDescription(intwDriverIndex,
[MarshalAs(UnmanagedType.VBByRefStr)] ref String lpszName, int cbName,
[MarshalAs(UnmanagedType.VBByRefStr)] ref String lpszVer, int cbVer);
The capGetDriverDescription function retrieves the version description of the capture driver. The wDriverIndex
parameter specifies the index of the capture driver. The index can range from 0 through 9.
The structures are: BITMAPINFO, BITMAPINFOHEADER and VIDEOHDR. The VIDEOHDR
structure is used by the callback function. It contains the buffered frame data. The BITMAPINFO
structure defines the dimensions and color information of a Windows-based, device-independent Bitmap
(DIB). The BITMAPINFOHEADER
structure contains information about the dimensions and color format of a DIB. In our case, it defines the video format (frame size and bits per frame).
Inside the Code
The main part of the solution is the WebCamera
class library, which consists of three classes:
WebCameraDevice
WebCameraEventArgs
WebCameraDeviceManager
WebCameraDevice
public WebCameraDevice(int frameWidth, int frameHeight, int preferredFPS,
int camID, int parentHwnd)
{
}
This initializes a new instance of the WebCameraDevice
object. Focus of preferredFPS
parameter: generally, Web cameras support a maximum of 30 FPS. The maximum FPS I could get on my A4Tech webcam was 20. Also, FPS depends on driver details. For example, enabling flicker slightly reduces FPS. Use the WebCamDeviceManager
class to get all available devices and their indices. The camID
parameter represents the selected device's index.
public void Start()
{
camHwnd = capCreateCaptureWindow("WebCam", 0, 0, 0, frameWidth,
frameHeight, parentHwnd, camID);
if (SendMessage(camHwnd, WM_CAP_DRIVER_CONNECT, 0, 0))
{
SendMessage(camHwnd, WM_CAP_SET_PREVIEW, 1, 0);
SendMessage(camHwnd, WM_CAP_SET_PREVIEWRATE, 34, 0);
SendBitmapMessage(camHwnd, WM_CAP_SET_VIDEOFORMAT,
Marshal.SizeOf(bInfo), ref bInfo);
frameThread = new Thread(new ThreadStart(this.FrameGrabber));
bStart = true;
frameThread.Priority = ThreadPriority.Lowest;
frameThread.Start();
}
}
The multithreading mechanism in WebCameraDevice
consists of the AutoResetEvent
object and frame-grabbing worker thread. Setting preferredFPS
to 0
allows the user to control the frame capturing process manually. The worker thread waits (WaitOne()
is called) until the user calls the AutoResetEvent
object's Set()
method (WaitHandle
receives a signal). Otherwise, WaitOne(..)
with the preferredFPSms
(1000 / preferredFPS)
parameter is called to wait for a defined amount of milliseconds.
After calling the Start()
method, the worker thread starts capturing frames to the buffer. The WebCameraDevice
object raises the OnCameraFrame
event that contains frame data in Bitmap
form.
private void FrameGrabber()
{
while (bStart)
{
SendMessage(camHwnd, WM_CAP_GRAB_FRAME_NOSTOP, 0, 0);
SendHeaderMessage(camHwnd, WM_CAP_SET_CALLBACK_FRAME, 0,
delegateFrameCallBack);
}
}
What happens in this block of code? The bStart
variable is a flag that turns to false
when the Stop()
method is called. While bStart
remains true
, the WM_CAP_GRAB_FRAME_NOSTOP
message fills the frame buffer with a single uncompressed frame from the capture device. Then a callback is made. We use the delegateFrameCallBack
variable instead of a direct callback function's name to avoid GC errors. Try replacing delegateFrameCallBack
with FrameCallBack
(callback function's name) and see what happens. The callback function looks like this:
private void <a class="code-string" name="<span">"FrameCallBack">FrameCallBack</a>(IntPtr hwnd, ref VIDEOHEADER hdr)
{
if (OnCameraFrame != null)
{
Bitmap bmp = new Bitmap(frameWidth, frameHeight, 3 *
frameWidth, System.Drawing.Imaging.PixelFormat.Format24bppRgb,
hdr.lpData);
OnCameraFrame(this, new WebCameraEventArgs(bmp));
}
if (preferredFPSms == 0)
{
autoEvent.WaitOne();
}
else
{
autoEvent.WaitOne(preferredFPSms, false);
}
}
As you can see, the function contains all Bitmap
converting, event raising and WaitHandler
operating stuff. That's it! The remaining methods are:
public void Set()
{
autoEvent.Set();
}
public void Stop()
{
try
{
bStart = false;
Set();
SendMessage(camHwnd, WM_CAP_DRIVER_DISCONNECT, 0, 0);
}
catch { }
}
public void ShowVideoDialog()
{
SendMessage(camHwnd, WM_CAP_DLG_VIDEODISPLAY, 0, 0);
}
How Does It Work?
We have a class library with all the necessary Web camera image capturing classes. First of all, we have to get the available VFW devices and display them to the user:
public FormMain()
{
InitializeComponent();
WebCameraDeviceManager camManager = new WebCameraDeviceManager();
cmbDevices.Items.AddRange(camManager.Devices);
cmbDevices.SelectedIndex = 0;
}
The start button and the OnCameraFrame
event handler's code:
private void btnStart_Click(object sender, EventArgs e)
{
camDevice = new WebCameraDevice
(320, 200, 0, cmbDevices.SelectedIndex, this.Handle.ToInt32());
camDevice.OnCameraFrame +=
new WebCameraFrameDelegate(camDevice_OnCameraFrame);
camDevice.Start();
}
void camDevice_OnCameraFrame(object sender, WebCameraEventArgs e)
{
ImageProcessing.Filters.Flip(e.Frame, false, true);
pictureBox.Image = e.Frame;
camDevice.Set();
}
Have you noticed the prefferedFPS
parameter's 0
value in WebCameraDevice
's constructor? That's why the Set()
method is called in the camDevice_OnCameraFrame
event handler. Do you remember what happens inside the camDevice
object? If not, check FrameCallBack
above.
Unexpected Image Flip
There was an unexpected vertical image flip. I haven't discovered why this happens yet. It happens only in the case of BITMAPINFOHEADER
's buffer conversion. Maybe there is a bug in the Bitmap
class. To avoid flipping, I've referred to a great Image Processing for Dummies with C# and GDI+ article by Christian Graus. A fast grayscale filter was found on Bob Powell's site.
History
- Release - 12 September, 2007
P.S.
I'd like to ask you to be lenient with the article because it's my first article on The Code Project. Please let me know if you have liked/disliked it or have any questions about it.