Capture Sample with DirectX and .NET

• Download demo project - 46.5 Kb
• Download source - 160 Kb

Introduction

This article tries to explain how to capture video and frames simultaneously. It is based on SampleGrabber program done by NETMaster and DirectX.Capture by Brian Low. The root of all of this was the amazing work of NETMaster, DirectShow.NET, SampleGrabber is just a sample application. I said amazing because I'm close to understand completely the other projects or classes, but DirectShow.NET is far of my knowledge.

Before begin with the explanation, I have to say that I am not an expert programmer, this work is one of the first serious programs that I have ever typed, so probably someone could find a best solution to do the same.

At this point you could probably guess that my mother tongue is not English, it is Spanish, sorry for my grammatical mistakes. I have write the code in Visual Studio .NET Spanish version, therefore the automatically generated comments are in Spanish, but my comments are in English.

The VB code

First of all, to run the demo program you will need to have installed NET Framework 1.1. check windows update if you don't have it.

Here I want to explain how the project works in VB, I will explain the sampleGrabber filter in the next section. The main project is CapSample, it has three forms; MW, AddCam and CW, and a module ModCap. Let's begin with the description.

MW stands for "Main Window", it's the initial form. When it is created, it create a new form, AddCam, that is responsible to the selection of an available camera. If OK button is clicked, AddCam will connect to the camera. The connection is done by using the capture class of DirectX.Capture, as I said before in the next section I will discuss my modifications to this class.

Once you have the camera connected (AddCam finished, if you click Cancel an error will happen), MW continues its initialization and set up the preview board (I am not sure about this word, in Spanish this control it's call "Panel"), then it add a handler to the FrameCaptureComplete event of the capture class and initialize the counters. Finishing with the creation of CW (It stands for "Configuration Window"), which is responsible of the configuration of the camera. By clicking OK ConfParamCam() will set the selected parameters, then it will set the capture directory (If it doesn't exist an error will happen), and capture a frame and render the preview stream by CaptureInformation.CaptureInfo.CaptureFrame().

Once you have configured the camera, you are able to capture frames by clicking the Frame button and start the video capture by clicking the Start button. When you click Stop, it stops the capture but it calls ConfParamCam() to set the previous selected parameters and PrepareCam() to increment the name of the captured file. And that is all I have to say about the VB code. Let's begin with the capture class.

Modifications in capture class of DirectX.Capture

My capture class it is a mixed class between capture class of Brian Low and SampleGrabberNET program of NETMaster.

In createGraph(), I have added the next code

AMMediaType media = new AMMediaType();
media.majorType= MediaType.Video;
media.subType = MediaSubType.RGB24;
media.formatType = FormatType.VideoInfo;
hr = sampGrabber.SetMediaType( media );
if( hr<0 ) Marshal.ThrowExceptionForHR( hr );

and

mediaEvt    = (IMediaEventEx)    graphBuilder;
baseGrabFlt = (IBaseFilter) sampGrabber;
hr = graphBuilder.AddFilter( baseGrabFlt, "DS.NET Grabber" );
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

The first block is to can use the filter and the second bock is the filter itself. The important things come with renderGraph(), before show you the code I have to say that all WDM devices have two PINs, capture and preview, sometimes only one (capture) but the smart tee filter convert the capture PIN into capture and preview, if it is needed Intelligent Connect will add it for us. So, I will use the capture PIN to capture video to a file and the preview PIN to capture frames. RenderGraph() is divided in two ifs, one to prepare all the filters to capture video to a file, it is controlled by wantCaptureRendered, and the other it is to render the preview stream, I have added the variable renderStream to avoid the normal functioning of the original class of Brian Low, because when baseGrabFlt is set-up I don�t know how to configure the cameras and when a capture was stopped, camera lost its configuration parameters, and I stop render until this parameters are updated with renderStream.

The first if is equal to the first if in Brian Low class, and the second looks like this.

// Render preview stream and launch the baseGrabFlt to capture frames

// ==================================================================

if ( wantPreviewRendered && renderStream && !isPreviewRendered )
{
    /// Render preview (video.PinPreview -> baseGrabFlt -> renderer)

    /// At this point intelligent connect is used, because my 

    /// webcams don't have a preview pin and

    /// a capture pin, so Smart Tee filter will be used. 

    /// I have tested it using GraphEdit.

    /// I can type hr = captureGraphBuilder.RenderStream( ref cat, 

    /// ref med, videoDeviceFilter, null, baseGrabFlt); 

    /// because baseGrabFlt is a transform filter, 

    /// like videoCompressorFilter.

    
    cat = PinCategory.Preview;
    med = MediaType.Video;
    hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
        videoDeviceFilter, baseGrabFlt, null ); 
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Get the IVideoWindow interface

    videoWindow = (IVideoWindow) graphBuilder;

    // Set the video window to be a child of the main window

    hr = videoWindow.put_Owner( previewWindow.Handle );
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Set video window style

    hr = videoWindow.put_WindowStyle( WS_CHILD | WS_CLIPCHILDREN 
       | WS_CLIPSIBLINGS);
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Position video window in client rect of owner window

    previewWindow.Resize += new EventHandler( onPreviewWindowResize );
    onPreviewWindowResize( this, null );

    // Make the video window visible, now that it is properly positioned

    hr = videoWindow.put_Visible( DsHlp.OATRUE );
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    hr = mediaEvt.SetNotifyWindow( this.Handle, 
         WM_GRAPHNOTIFY, IntPtr.Zero );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );

    isPreviewRendered = true;
    didSomething = true;

    // Begin Configuration of SampGrabber    <<<<<<-----------------

                
    AMMediaType media = new AMMediaType();
    hr = sampGrabber.GetConnectedMediaType( media );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );
    if( (media.formatType != FormatType.VideoInfo) || 
              (media.formatPtr == IntPtr.Zero) )
        throw new NotSupportedException( "Unknown Grabber Media Format" ); 

    videoInfoHeader = (VideoInfoHeader) Marshal.PtrToStructure( 
               media.formatPtr, typeof(VideoInfoHeader) );
    Marshal.FreeCoTaskMem( media.formatPtr ); 
    media.formatPtr = IntPtr.Zero;

    hr = sampGrabber.SetBufferSamples( false );
    if( hr == 0 )
        hr = sampGrabber.SetOneShot( false );
    if( hr == 0 )
        hr = sampGrabber.SetCallback( null, 0 );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );    
                
    // Finish Configuration of SampGrabber    <<<<<<----------------

}
            
if ( didSomething )
    graphState = GraphState.Rendered;

In this four lines is when the baseGrabFlt is set-up, note that I use baseGrabFlt as a transform filter.

cat = PinCategory.Preview;
med = MediaType.Video;
hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
    videoDeviceFilter, baseGrabFlt, null ); 
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

About the configuration of sampleGrabber, I only comment next lines.

hr = sampGrabber.SetBufferSamples( false );
if( hr == 0 )
    hr = sampGrabber.SetOneShot( false );
if( hr == 0 )
    hr = sampGrabber.SetCallback( null, 0 );
if( hr < 0 )
    Marshal.ThrowExceptionForHR( hr );

SampleGrabber has three different methods to capture a frame, I choose SetCallBack because I found it more useful, sampleGrabber will capture a frame when a call to a function was done.

We need more things to complete the capture of a frame, and it is done by the next block of code.

void OnCaptureDone()
{
    int hr;
    if( sampGrabber == null )
        return;
    hr = sampGrabber.SetCallback( null, 0 );

    int w = videoInfoHeader.BmiHeader.Width;
    int h = videoInfoHeader.BmiHeader.Height;
    if( ((w & 0x03) != 0) || (w < 32) || (w > 4096) 
            || (h < 32) || (h > 4096) )
        return;
    int stride = w * 3;

    GCHandle handle = GCHandle.Alloc( savedArray, GCHandleType.Pinned );
    int scan0 = (int) handle.AddrOfPinnedObject();
    scan0 += (h - 1) * stride;
    Bitmap b = new Bitmap( w, h, -stride, 
        PixelFormat.Format24bppRgb, (IntPtr) scan0 );
    handle.Free();
    savedArray = null;
    ImageCaptured.Image = b;

    //Launch the event


    FrameCaptureComplete(ImageCaptured);

    return;
        
}

protected override void WndProc( ref Message m )
{
    if( m.Msg == WM_GRAPHNOTIFY )
    {
        if( mediaEvt != null )
            OnGraphNotify();
                return;
    }
    base.WndProc( ref m );
}

// graph event (WM_GRAPHNOTIFY) handler

void OnGraphNotify()
{
    DsEvCode    code;
    int p1, p2, hr = 0;
    do
    {
        hr = mediaEvt.GetEvent( out code, out p1, out p2, 0 );
        if( hr < 0 )
            break;
        hr = mediaEvt.FreeEventParams( code, p1, p2 );
    }
    while( hr == 0 );
}

int ISampleGrabberCB.SampleCB( double SampleTime, 
    IMediaSample pSample )
{
    return 0;
}
        
int ISampleGrabberCB.BufferCB(double SampleTime, 
    IntPtr pBuffer, int BufferLen )
{
    if( captured || (savedArray == null) )
    {
        return 0;
    }
    captured = true;
    bufferedSize = BufferLen;
    if( (pBuffer != IntPtr.Zero) && (BufferLen > 1000)
            && (BufferLen <= savedArray.Length) )
        Marshal.Copy( pBuffer, savedArray, 0, BufferLen );
    try
    {
        this.BeginInvoke( new CaptureDone( this.OnCaptureDone ) );
    }
    catch (ThreadInterruptedException e)
    {
        MessageBox.Show(e.Message);
    }
    catch (Exception we)
    {
        MessageBox.Show(we.Message);
    }
    return 0;
}

Beginning at the end, ISampleGraberCB.BufferCB(...) is a function that is filling a buffer, and when the buffer is full it have the frame, and call OnCaptureDone() that is the function that create the image. When the image was built it issues an event to send it to the receiver of that event.

ISampleGrabberCB.SampleCB(...) have to be defined instead it was not be used. For me the other two functions are black magic, I can guess what they are doing but I don't understand it.

And finally the method invoked to capture a frame.

public void CaptureFrame()
{
    int hr;

    if(firstFrame)
    {
        assertStopped();

        // Re-render the graph (if necessary)

        renderStream = true;
        renderGraph();

        // Start the filter graph: begin capturing

        hr = mediaControl.Run();
        if ( hr != 0 ) Marshal.ThrowExceptionForHR( hr ); 

        firstFrame = false;
    }

    captured = false;

    if(savedArray == null )
    {
        int size = videoInfoHeader.BmiHeader.ImageSize;
        if( (size<1000) || (size > 16000000) )
            return;
        savedArray = new byte[ size + 64000];
    }
    hr = sampGrabber.SetCallback( this, 1 );
}

firstFrame is to avoid execute more than once next line.

hr = mediaControl.Run()

I think the description is completed. I hope it will be useful to you :-)

System Requirements

I have been working with an AMDXP@1500, 512Mb RAM and WIN2k. I have installed DirectX 9.0 but, I think, it could work with DirectX 8.1.

As cameras I have been using a Philips ToUcam Pro and a Creative Video Blaster Web Cam 5 without problems. I have also installed a Pinacle PCTV Pro and it doesn't work capture a video to a file. I don't know why, but at the moment I don't have enough time to search the source of the problem.

Feedback and Improvements

I will try to keep track of the forum, if you need help I will be glad to give you a hand. I think we have to share code to improve the performance of our programs, and to help people do new things, for example, if I have not read the code of Brian Low I would never have written this code. Thank you!