Capture image from a streaming URL using different ISampleGrabber modes

• Download demo project- 34 KB

Introduction

This article explains how to grab a frame from a streaming URL (like a stream from WME) and save it as a BMP using DirectShow. In this article, I am using the IBase filter for the WM ASF Reader (network reader) for reading the network resource, and the ISampleGrabber filter to grab the bitmap frame. Here, I have tried to explain two methods (SetCallBack and GetCurrentBuffer) to grab a bitmap using the ISampleGrabber filter.

Background

In my recent project, I met with this same problem, capturing a snapshot from a WME streaming URL. I searched about it in the web and saw a lot of articles to capture images from a movie file or something like that, but article about how to capture an image from a streaming URL. I implemented this using some references from MSDN and the web. Initially, I implemented a solution by using the IFileSourceFilter and the GetCurrentBuffer method in the ISampleGrabber interface. That was enough for my needs. After that, I tried the CallBack (SetCallBack) in the ISampleGrabber interface for a different experience. I have explained these two modes in here. I will try to explain what I have learned and implemented. I hope the sample code will support someone in solving their problem.

Setting Up Your Visual Studio Project

You need to add header files from the Windows Platform SDK and the DShow base classes to your include path. The project has to be linked with Strmbase.lib.

#include "qedit.h"   // SampleGrabber filter

#include "atlbase.h" // for using atl support

#include "dshow.h"   // DirectShow header

Using the Code

The following DShow interfaces will be used in this article...

IGraphBuilder *pGraph = NULL; //Graph Manager 
IMediaControl *pControl = NULL; //Media Control for run the graph
IMediaEvent *pEvent = NULL; //Media Event interface for capture the media events
IBaseFilter *pWMASFReader = NULL; //IBase Filter for WM ASF READER (network reader) 
IPin *pStreamOut = NULL,*pStreamRender = NULL; //IPin interfaces for connecting filters 
IFileSourceFilter *pIFileSourceFilter = NULL; //for specifiying network target....

First, initialize COM and create the filter graph manager.

HRESULT hr = CoInitialize(NULL); 

if(FAILED(hr)) 
    return FALSE ; 

hr = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER, 
                      IID_IGraphBuilder, (void **)&pGraph); 

if(FAILED(hr)) 
    return FALSE;

Create the WM ASF Reader filter for network reading, and add it to the graph. Then, query for the file source filter to specify the network location, and load the specified network location (e.g.: http://host:port) using the IFileSourceFilter::Load() method. It will load the desired network source; if the network source is not found or is invalid, then it will return an HRESULT value with the result code. m_strURL is a CString variable that contains the desired network location.

hr = CoCreateInstance(CLSID_WMAsfReader,NULL,CLSCTX_INPROC_SERVER, 
                      IID_IBaseFilter, (void **) &pWMASFReader); 

if (SUCCEEDED(hr)) 
{ 
    hr = pGraph->AddFilter(pWMASFReader,L"WM ASF Reader"); 

    if (SUCCEEDED(hr)) 
    { 
        hr = pWMASFReader->QueryInterface(IID_IFileSourceFilter, 
                                         (void **) &pIFileSourceFilter); 

        if (SUCCEEDED(hr)) 
        { 
            // load network source 
            hr = pIFileSourceFilter->Load(m_strURL.AllocSysString(), NULL); 
        } 
    } 
} 
if(FAILED(hr))
   return FALSE;

Now we have the network source with us. We need to create the Sample Grabber filter and get the Sample Grabber interface. Create the Sample Grabber filter, add it to the graph, and query for the ISampleGrabber (ISampleGrabber *m_pGrabber).

// Create the Sample Grabber.
IBaseFilter *pGrabberF = NULL;
hr = CoCreateInstance(CLSID_SampleGrabber, NULL, CLSCTX_INPROC_SERVER,
                      IID_IBaseFilter, (void**)&pGrabberF);
if (FAILED(hr))
{
    // Return an error.
    return FALSE;
}
hr = pGraph->AddFilter(pGrabberF, L"Sample Grabber");
if (FAILED(hr))
{
    // Return an error.
    return FALSE;
}

pGrabberF->QueryInterface(IID_ISampleGrabber, (void**)&m_pGrabber);

We need to specify the media type for the connection on the input pin of the Sample Grabber. For that, we will use the AM_MEDIA_TYPE structure.

AM_MEDIA_TYPE mt;
ZeroMemory(&mt, sizeof(AM_MEDIA_TYPE));
mt.majortype = MEDIATYPE_Video;
mt.subtype = MEDIASUBTYPE_RGB24;
hr = m_pGrabber->SetMediaType(&mt);

You can add a null renderer filter in your graph to prevent the preview window from being displayed, if you need. The null renderer filter is a renderer that discards every sample it receives, without displaying or rendering the sample data. Otherwise, you can query IVideoWindow from the graph and can handle the popup window.

IBaseFilter    *pNullRenderer;
hr = CoCreateInstance (CLSID_NullRenderer, NULL, CLSCTX_INPROC_SERVER,
                       IID_IBaseFilter, (void **)&pNullRenderer);
hr = pGraph->AddFilter(pNullRenderer, L"Null Renderer");

Now we will connect the network reader filter to the grabber filter. The Sample Grabber is a transform filter, so the output pin must be connected to another filter. Often, you may simply want to discard the samples after you are done with them. In that case, connect the Sample Grabber to the null renderer filter, which discards the data that it receives.

hr = ConnectFilters(pGraph, pWMASFReader, pGrabberF);

The following function will connect the first filter to the second filter:

HRESULT CVideoImageCapDlg::ConnectFilters(IGraphBuilder *pGraph, 
        IBaseFilter *pFirst, IBaseFilter *pSecond)
{
     IPin *pOut = NULL, *pIn = NULL;
    HRESULT hr = GetPin(pSecond, PINDIR_INPUT, &pIn);
    if (FAILED(hr)) return hr;
    // The previous filter may have multiple outputs, so try each one!
    IEnumPins  *pEnum;
    pFirst->EnumPins(&pEnum);
    while(pEnum->Next(1, &pOut, 0) == S_OK)
    {
            PIN_DIRECTION PinDirThis;
            pOut->QueryDirection(&PinDirThis);
            if (PINDIR_OUTPUT == PinDirThis)
            {
                hr = pGraph->Connect(pOut, pIn);
                if(!FAILED(hr))
                {
                       break;
                }
            }
            pOut->Release();
    }
    pEnum->Release();
    pIn->Release();
    pOut->Release();
    return hr;
}

Then, get the output pin and render the stream.

IPin * pGrabOutPin=NULL;
hr= GetPin( pGrabberF,PINDIR_OUTPUT,&pGrabOutPin);

HRESULT CVideoImageCapDlg::GetPin(IBaseFilter *pFilter, PIN_DIRECTION PinDir, IPin **ppPin)
{
    IEnumPins  *pEnum;
    IPin       *pPin;
    pFilter->EnumPins(&pEnum);
    while(pEnum->Next(1, &pPin, 0) == S_OK)
    {
         PIN_DIRECTION PinDirThis;
         pPin->QueryDirection(&PinDirThis);
         if (PinDir == PinDirThis)
         {
                pEnum->Release();
                *ppPin = pPin;
                return S_OK;
         }
         pPin->Release();
    }
    pEnum->Release();
    return E_FAIL;  
}


hr = pGraph->Render(pGrabOutPin);
if( FAILED( hr ) )
{
    AfxMessageBox("Could not render grabber output pin\r\n");
    return FALSE;
}

Now query the Media control for running the graph and the Media event for retrieving event notifications.

hr = pGraph->QueryInterface(IID_IMediaEvent, (void **)&pEvent); 
if(FAILED(hr)) 
    return FALSE; 

hr = pGraph->QueryInterface(IID_IMediaControl, (void **)&pControl); 
if(FAILED(hr)) 
return FALSE ;

The Sample Grabber operates in one of two modes:

1. Buffering mode makes a copy of each sample before delivering the sample downstream.
2. Callback mode invokes an application-defined callback function on each sample.

Call the ISampleGrabber::SetOneShot method with the value FALSE. This causes the Sample Grabber to continue the running state of the graph after it receives the first media sample. And, ISampleGrabber::SetBufferSamples with the value TRUE to specify whether to copy sample data into a buffer as it goes through the filter. So that you can get the copied buffer by calling ISampleGrabber::GetCurrentBuffer.

// Set one-shot mode false and  buffering.
hr = m_pGrabber->SetOneShot(FALSE);
hr = m_pGrabber->SetBufferSamples(TRUE);

If everything goes fine, then we can run the graph.

long evCode=0;

hr=pControl->Run(); // Run the graph.

hr=pEvent->WaitForCompletion(INFINITE, &evCode); // Wait till its done.
//please be carefull when using INFINITE, 
//this may block the entire app indefinitely.
//It will cause the application hang up.
Sleep(1000);

If the graph is running perfectly, then we can retrieve the current buffer by using the ISampleGrabber::GetCurrentBuffer() method. The SaveBufferToBitmap(mt) function will grab the buffer using the ISampleGrabber::GetCurrentBuffer() method and will save it as BMP.

BOOL CVideoImageCapDlg::SaveBufferToBitmap(AM_MEDIA_TYPE mt)
{
    // Find the required buffer size.
    long cbBuffer = 0;
    HRESULT hr = m_pGrabber->GetCurrentBuffer(&cbBuffer, NULL);

    switch(hr)
    {
        case E_INVALIDARG:
            AfxMessageBox("E_INVALIDARG");
            break;

        case E_OUTOFMEMORY:
            AfxMessageBox("E_OUTOFMEMORY");
            break;

        case E_POINTER:
            AfxMessageBox("E_POINTER");
            break;

        case VFW_E_NOT_CONNECTED:
            AfxMessageBox("VFW_E_NOT_CONNECTED");
            break;

        case VFW_E_WRONG_STATE:
            AfxMessageBox("VFW_E_WRONG_STATE");
            break;
    }

    if(FAILED(hr))
        return FALSE;

    char *pBuffer = new char[cbBuffer];
    if (!pBuffer) 
    {
        // Out of memory. Return an error code.
        return FALSE;
    }
    hr = m_pGrabber->GetCurrentBuffer(&cbBuffer, (long*)pBuffer);


    ZeroMemory(&mt, sizeof(AM_MEDIA_TYPE));
    hr = m_pGrabber->GetConnectedMediaType(&mt);
    if (FAILED(hr))
    {
        // Return error code.
        return FALSE;
    }
    // Examine the format block.
    VIDEOINFOHEADER *pVih;
    if ((mt.formattype == FORMAT_VideoInfo) && 
        (mt.cbFormat >= sizeof(VIDEOINFOHEADER)) &&
        (mt.pbFormat != NULL) ) 
    {
        pVih = (VIDEOINFOHEADER*)mt.pbFormat;
    }
    else 
    {
        return FALSE;
    }

     //
     // Save image data as Bitmap.
     // This is just to make this sample easily understandable.
     //
     HANDLE fh;
     BITMAPFILEHEADER bmphdr;
     DWORD nWritten;

     memset(&bmphdr, 0, sizeof(bmphdr));

     bmphdr.bfType = ('M' << 8) | 'B';
     bmphdr.bfSize = sizeof(bmphdr) + sizeof(BITMAPINFOHEADER) + cbBuffer;
     bmphdr.bfOffBits = sizeof(bmphdr) + sizeof(BITMAPINFOHEADER);

     fh = CreateFile(m_strSaveTo,
     GENERIC_WRITE, 0, NULL,
     CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
     WriteFile(fh, &bmphdr, sizeof(bmphdr), &nWritten, NULL);
     WriteFile(fh,
     &pVih->bmiHeader,
     sizeof(BITMAPINFOHEADER), &nWritten, NULL);
     WriteFile(fh, pBuffer, cbBuffer, &nWritten, NULL);
     CloseHandle(fh);

     free(pBuffer);

     return TRUE;
}

We can also grab the media sample by invoking the callback function by setting the ISampleGrabber->SetCallback() method. FrameGrabCallback is a class derived from ISampleGrabberCB. What follows is an example of the callback class. Note that the class implements IUnknown, which it inherits through the ISampleGrabber interface, but it does not keep a reference count. This is safe because the application creates the object on the stack, and the object remains in scope throughout the lifetime of the filter graph. All of the work happens in the BufferCB method, which is called by the Sample Grabber whenever it gets a new sample. In the following example, the method writes the bitmap to a file:

//
// Callback class for Sample Grabber filter.
//
class FrameGrabCallback : public ISampleGrabberCB
{
public :

 // Fake out any COM ref counting
    //
  STDMETHODIMP_(ULONG) AddRef() { return 2; }
  STDMETHODIMP_(ULONG) Release() { return 1; }
  STDMETHODIMP QueryInterface(REFIID iid, void** ppv)
  {
    if (NULL == ppv)
      return E_POINTER;
  
    *ppv = NULL;
    if (IID_IUnknown == iid)
    {
      *ppv = (IUnknown*)this;
      AddRef();
      return S_OK;
    }
    else 
    if (IID_ISampleGrabberCB == iid)
    {
      *ppv = (ISampleGrabberCB*)this;
      AddRef();
      return S_OK;
    }
  
    return E_NOINTERFACE;
  }
    
  // Constructor/destructor
  FrameGrabCallback() 
  {
      framenum=0;
  }
    
  ~FrameGrabCallback() {}

public:
  // These will get set by the main thread below. We need to
  // know this in order to write out the bmp
  long Width;
  long Height;
  long framenum;
  CString strPath;

  STDMETHODIMP SampleCB(double n,IMediaSample *pms)
  {
    return 0;
  }


    // The sample grabber is calling us back on its deliver thread.
    // This is NOT the main app thread!
    //
    STDMETHODIMP BufferCB( double SampleTime, BYTE * pBuffer, long BufferSize )
    {
        //
        // Convert the buffer into a bitmap
        // strPath ="path"; eg : "c://Test.bmp"

        TCHAR szFilename[MAX_PATH];
        if(strPath=="")
        {
            wsprintf(szFilename, TEXT("bitmap%ld.bmp\0"),  framenum );
        }
        else
        {
            wsprintf(szFilename, strPath ,  framenum );
        }
        
        
        framenum++;
        // Create a file to hold the bitmap
        HANDLE hf = CreateFile(szFilename, GENERIC_WRITE, FILE_SHARE_READ, 
                               NULL, CREATE_ALWAYS, NULL, NULL );

        if( hf == INVALID_HANDLE_VALUE )
        {
            _tprintf( TEXT("INVALID_HANDLE_VALUE\r\n"));
        }

        /*_tprintf(TEXT("Found a sample at time %ld ms\t[%s]\r\n"), 
                 long(SampleTime*1000), szFilename );*/

        // Write out the file header
        //
        BITMAPFILEHEADER bfh;
        memset( &bfh, 0, sizeof( bfh ) );
        bfh.bfType = 'MB';
        bfh.bfSize = sizeof( bfh ) + BufferSize + sizeof( BITMAPINFOHEADER );
        bfh.bfOffBits = sizeof( BITMAPINFOHEADER ) + sizeof( BITMAPFILEHEADER );

        DWORD Written = 0;
        WriteFile( hf, &bfh, sizeof( bfh ), &Written, NULL );

        // Write the bitmap format
        //
        BITMAPINFOHEADER bih;
        memset( &bih, 0, sizeof( bih ) );
        bih.biSize = sizeof( bih );
        bih.biWidth = Width;
        bih.biHeight = Height;
        bih.biPlanes = 1;
        bih.biBitCount = 24;

        Written = 0;
        WriteFile( hf, &bih, sizeof( bih ), &Written, NULL );

        // Write the bitmap bits
        //
        Written = 0;
        WriteFile( hf, pBuffer, BufferSize, &Written, NULL );

        CloseHandle( hf );

        return 0;
    }
};

After the creation of this class, please use the following code before running the graph. Only use one of the two methods (callback or get current buffer) at a time.

// Set the callback, so we can grab the one sample
//Also set the video width and height to grab the frame.

FrameGrabCallback   m_FrameGrabCallback;

m_FrameGrabCallback.Width=320;
m_FrameGrabCallback.Height=240;
m_FrameGrabCallback.framenum=1;

hr = m_pGrabber->SetCallback( &m_FrameGrabCallback, 1 );//Set the callback method

After everything is done, we can stop the graph, unsubscribe the callback function, and release all the resources.

m_pGrabber->SetCallback(NULL,1);
pControl->Stop();
pControl->Release();
m_pGrabber->Release();
pEvent->Release();
pGraph->Release();
pGrabberF->Release();

Reference

• MSDN: Using the Sample Grabber

History

• First version.