Creating Custom DirectShow SampleGrabber Filter for Windows Mobile

• Download source - 7.67 KB
• Download sample C# client project with modified CameraCaptureDLL - 32.91 KB

Introduction

This article explains how to write your custom DirectShow SampleGrabber filter for Windows Mobile.

Background

In a recent project of mine, I needed to do some real-time video analysis in .NET CF. However the .NET API only allows for taking still images or recording videos, but it offers no way to access the video frame buffer so that I could parse the frames from the camera on the fly. So I decided to use the DirectShow API, which offers much better control of the video stream flow, but it still lacks the ISampleGrabber interface which is available under the complete DirectShow library for Windows. The remaining option was to write a custom DirectShow filter, which implements the ISampleGrabber interface and allows the developer to get access to the video buffer data. In this article, I will try to explain what I learned in the process.

You could use the sample code as a starting point to write your own filter or take the ready SampleGrabber filter and use it in your project. There are multiple references online which I used in the project, but as far as I am aware there is no complete "How to" on how to solve this problem.

Setting Up Your Visual Studio Project

Before your set up your Visual Studio project, you will need to install Windows Mobile SDK, Windows CE 5.0 and Windows CE 5.0 Platform builder which contains the BaseClasses library. In general, you don't necessarily use the BaseClasses, but it definitely makes your job a lot easier so I took advantage.

First create a Smart Devices DLL project with ATL support. You also need to add the header files from the Windows Mobile SDK and Platform Builder to your include path.

Writing the Filter

In this example, we will use a TransInPlaceFilter, which is a simplified version of the TransformFilter. We don't need to alter the data in any way, just to pass it to the client application. So we create a CSampleGrabber class, which also implements our custom interface with any additional functions. I added just the RegisterCallback function, which passes a function pointer from the client app. This function will be called whenever a MediaSample (a video frame) is passed through the filter, so the client can copy the data and do some processing.

// define the filter class
class CSampleGrabber : 
public CTransInPlaceFilter, public ISampleGrabber
{
  private:
  MANAGEDCALLBACKPROC callback;
  long m_Width;
  long m_Height;
  long m_SampleSize;
  long m_Stride;

  public:
  // instantiation
  CSampleGrabber( IUnknown * pOuter, HRESULT * phr, BOOL ModifiesData );
  ~CSampleGrabber();
  static CUnknown *WINAPI CreateInstance(LPUNKNOWN punk, HRESULT *phr);

  // IUnknown
  DECLARE_IUNKNOWN;
  STDMETHODIMP NonDelegatingQueryInterface(REFIID riid, void ** ppv);

  // CTransInPlaceFilter
  HRESULT CheckInputType(const CMediaType *pmt);
  HRESULT SetMediaType(PIN_DIRECTION direction, const CMediaType *pmt);
  HRESULT Transform(IMediaSample *pMediaSample);
  HRESULT DecideBufferSize(IMemAllocator *pAlloc, ALLOCATOR_PROPERTIES *pProperties);
  HRESULT GetMediaType(int iPosition, CMediaType *pMediaType);
  HRESULT CheckTransform(const CMediaType *mtIn, const CMediaType *mtOut) {
    return NOERROR; 
    }

  // ISampleGrabber
  STDMETHODIMP RegisterCallback(MANAGEDCALLBACKPROC mdelegate);
};

You could implement all interfaces like any other filter. The interesting parts are the Transform and RegisterCallback functions:

// Set the client callback
STDMETHODIMP
CSampleGrabber::RegisterCallback( MANAGEDCALLBACKPROC mdelegate ) 
{
    // Set pointer to managed delegate
    callback = mdelegate;
    return S_OK;
}

// Get the pointer to the raw data and pass it to the applications
HRESULT 
CSampleGrabber::Transform(IMediaSample *pMediaSample)
{
    long Size = 0;
    BYTE *pData;

    if ( !pMediaSample )
        return E_FAIL;

    // Get pointer to the video buffer data
    if( FAILED(pMediaSample->GetPointer(&pData)) ) 
        return E_FAIL;
    Size = pMediaSample->GetSize();

    // invoke managed delegate
    if ( callback )
        callback(pData,Size);

    return S_OK;
}

Using the Code

You can use the SampleGrabber filter directly in your DirectShow applications exactly like any other filter. It is important to register the DLL before trying to instantiate it.

// Create and initialize the SampleGrabber filter
CoCreateInstance( CLSID_SampleGrabber, NULL, CLSCTX_INPROC, 
		IID_IBaseFilter, (void**)&pSampleGrabber );
m_pFilterGraph->AddFilter( pSampleGrabber, FILTERNAME );

// Get a pointer to the ISampleGrabber interface
pSampleGrabber->QueryInterface( IID_ISampleGrabber, (void**)&m_pISampleGrabber );

// Register the client callback
if ( m_pISampleGrabber )
  m_pISampleGrabber->RegisterCallback( &CGraphBuilder::OnSampleProcessed );

When the client receives the frame sample, it should copy it to local buffer as soon as possible and return. This allows the filter to carry on and not wait for the client to process the data, which will considerably slow down the whole graph.

You could also make a C++ DLL which creates the filter graph and manages it and calls it through P/Invoke from your .NET CF applications. Creating filter graphs in .NET CF directly should be a bit trickier, as the .NET CF lacks C++. NET support, but maybe possible.

Online References

• A great DirectShow and COM article which goes into detail on writing DirectShow Transform filters
• Great DirectShow information from RadScorpion's Blog
• Windows Mobile Developer Center from Microsoft
• Creating a Transform Filter from MSDN
• How To Get Data from a Microsoft DirectShow Filter Graph from MSDN

History

• 23.08.2008 Article created
• 26.08.2008 v1.1 Created.
  Fixes:
  • Release now builds OK, definition of NonDelegatingRelease added
  • Included C++ and C# client applications which show how to use the filter
  • The sample code now compiles under WM SDK 5
• 28.11.2008
  • Added more sample code: CameraCaptureDLL project includes the CameraCapture sample and the builtin SampleGrabber filter. The TestCameraCapture project is a sample C# application that uses the filter to get the real-time frame data.
  • Note: The sample code demonstrates the basic concepts, but there are a couple of issues with it. For example, it might get stuck when trying to stop the filter graph. This is due to the fact that the filter is processing data while you try to do that (sorry but I didn't have time to fix that). Although buggy, you should be able to get the camera data.