DirectShow Filters Development Part 2: Live Source Filter

• Download source code and demo - 9.78 KB

Introduction

A vast majority of software and hardware vendors working in the digital video field supply a source filter capable of getting frames from some hardware device like video acquisition card or IP camera. So our mission as software developers is as simple as build a filter graph, add this source filter, and render it. However, not all applications and hardware devices are built with DirectShow in mind, and when you need to use them, you will probably build a source filter of your own which will wrap the API of that application or hardware provider.

To make this task easy, I decided to make a generic source filter which exposes an interface for pushing downstream RGB samples of predefined size and frame rate, and can be used for any custom frame input scenario (I hope :)).

Before going any further, take a look at part 1 of this series as the filter development prerequisites, filter registration, and debugging are same for all filter types.

Source filters

Source filter is the most important filter in the graph as it dictates the media size, frame rate, and media format. Each filter graph contains at least one source filter which can produce several media streams not necessarily of the same type. For example, a media splitter source filter can produce video, audio, and text (subtitles) streams each of which is pushed to downstream filters through its own pin. Usually, the information regarding the streams is contained inside a media file. When there is no media file, this information resides inside a source filter, or inside its output pins.

When building source filters, you basically have two choices:

1. Build a source filter which inherits from the CSource base class, and add a nested pin class which inherits from CSourceStream and is responsible for the actual frame creation and delivery. CSourceStream inherits from the CAMThread class which encapsulates the background thread responsible for delivering media samples to the downstream filter through its input pin. These kinds of filters are called push source filters, and they are well documented on MSDN and have numerous examples like the PushSource and Bouncing Ball samples. In addition, there are some articles here on CodeProject describing how to build these filters. Therefore, I will not cover this topic.
2. Instead, I will write about the less documented Live Source filters. To create a live source filter, use CBaseFilter as the filter base class, and its pin class should inherit from CBaseOutputPin.

class CLiveSourceStream;

class CLiveSource : public CBaseFilter, public IAMFilterMiscFlags
{
public:
   DECLARE_IUNKNOWN;

   CLiveSource(LPUNKNOWN pUnk, HRESULT* phr);
   virtual ~CLiveSource(void);

   static CUnknown* WINAPI CreateInstance(LPUNKNOWN pUnk, HRESULT *phr);
   STDMETHODIMP NonDelegatingQueryInterface(REFIID riid, void ** ppv);

   int GetPinCount();
   CBasePin* GetPin(int n);

   virtual ULONG STDMETHODCALLTYPE GetMiscFlags( void)
   {
          return AM_FILTER_MISC_FLAGS_IS_SOURCE;
   }

private:
   CLiveSourceStream* m_pOutputPin;
   CCritSec m_critSec;
};

CBaseFilter is an abstract class and you need to implement its two pure virtual functions:

int CLiveSource::GetPinCount()
{
   CAutoLock cAutoLock(&m_critSec);

   return 1;
}

CBasePin* CLiveSource::GetPin(int n)
{
   CAutoLock cAutoLock(&m_critSec);

   return m_pOutputPin;
}

GetPinCount returns 1 since there is only one output pin which pushes video samples provided from another context using the ILiveSource interface. GetPin returns a pointer to that only pin.

According to MSDN documentation, a live source filter delivers a media sample at an inconstant rate. The filter is considered as a live source filter if it implements the IAMFilterMiscFlags interface and the filter's output pin implements the IAMPushSource interface.

class CLiveSourceStream : public CBaseOutputPin, 
                 public ILiveSource, public IAMPushSource
{
public:

   DECLARE_IUNKNOWN;

   CLiveSourceStream(CBaseFilter *pFilter, CCritSec *pLock, HRESULT *phr);
   virtual ~CLiveSourceStream();

   // CBaseOutputPin overrides
   virtual HRESULT GetMediaType(int iPosition, CMediaType* pmt);
   virtual HRESULT CheckMediaType(const CMediaType *pmt);
   virtual HRESULT DecideBufferSize(IMemAllocator *pAlloc, 
                   ALLOCATOR_PROPERTIES *ppropInputRequest);

   // ILiveSource members
   virtual HRESULT AddFrame(HBITMAP hBmp);
   virtual HRESULT AddFrame(BYTE* pBuffer, DWORD size);
   virtual HRESULT SetFrameRate(int frameRate);
   virtual HRESULT SetBitmapInfo(BITMAPINFOHEADER& bInfo);

   // IAMPushSource members
   virtual STDMETHODIMP GetPushSourceFlags(ULONG *pFlags);
   virtual STDMETHODIMP SetPushSourceFlags(ULONG Flags);
   virtual STDMETHODIMP SetStreamOffset(REFERENCE_TIME rtOffset);
   virtual STDMETHODIMP GetStreamOffset(REFERENCE_TIME *prtOffset);
   virtual STDMETHODIMP GetMaxStreamOffset(REFERENCE_TIME *prtMaxOffset);
   virtual STDMETHODIMP SetMaxStreamOffset(REFERENCE_TIME rtMaxOffset);
   virtual STDMETHODIMP GetLatency(REFERENCE_TIME *prtLatency);

   virtual STDMETHODIMP Notify(IBaseFilter * pSender, Quality q);

private:
   HRESULT GetPixelData(HBITMAP hBmp, BYTE** ppData, int* pSize);
   HRESULT GetMediaSample(IMediaSample** ppSample);

private:
   BITMAPINFOHEADER m_bmpInfo;
   int m_frameRate;
   REFERENCE_TIME m_rtFrameRate; 
   REFERENCE_TIME m_lastFrame; 
};

Although IAMPushSource is implemented here, it is pretty much useless since there is only one output pin and no synchronization is needed. If, for example, you would want to have a second pin with a different sample rate, then you should implement this interface as described in here.

The ILiveSource filter has four methods:

struct ILiveSource : IUnknown
{
   // Adds bitmap to the video sequence
   virtual HRESULT AddFrame(HBITMAP hBmp) PURE;

   // Adds pixel data buffer to the video sequence
   virtual HRESULT AddFrame(BYTE* pBuffer, DWORD size) PURE;

   // Set the video frame info.
   // Default value is width = 704,
   // height = 576 (4CIF) and 32 bits per pixel
   virtual HRESULT SetBitmapInfo(BITMAPINFOHEADER& bInfo) PURE;

   // Set the expected frame rate of the video.
   // Value should be in range of [0,30]
   // Default value is 0
   virtual HRESULT SetFrameRate(int frameRate) PURE;
};

The first two methods are responsible for passing frames into a running filter graph. The first method uses the GetObject API (described below) to get the pixel data and other bitmap parameters. The second AddFrame override takes a pointer to the pixel data buffer and the size of the buffer to prevent overflows. The size is checked against the biSizeImage member of the BITMAPINFOHEADER structure to determine whether it is valid.

By default, BITMAPINFOHEADER is set to 4CIF frames (704 x 576) and 32 bits per pixel; however, you can set it using the SetBitmapInfo method.

SetFrameRate is used to set the start and end time for each media sample. Using these values, the renderer filter schedules each sample for display and monitors quality control. By default, the frame rate is set to 0, which means no scheduling will take place and each media sample will be rendered immediately after arriving to the input pin of the renderer.

Media samples

IMediaSample is the transport unit used between pins of DirectShow filters. Source filters are responsible for creating them, filling the data, and passing it downstream. A transform filter performs some operations on that data, and renderer filters render them either on screen, network, or another location. Each media sample contains a memory buffer which holds the actual sample data - in the case of a video, it will usually have pixel data either raw or encoded. Each sample also has reference times which govern the playback and quality control messages sent in the reverse direction - from renderer filter to the source filter or whoever can handle such types of messages.

When using the CBaseOutputPin class, it your responsibly to get the media sample from the memory allocator, fill it with data, set time stamps, set the sample size, and deliver the sample to the input pin of the connected downstream filter. Hence, the AddFrame method implementation looks as follows:

HRESULT CLiveSourceStream::AddFrame(HBITMAP hBmp)
{
   CAutoLock cAutoLock(m_pLock);

   IMediaSample* pSample = NULL;
   BYTE* pData = NULL;

   HRESULT hr = GetMediaSample(&pSample);
   if(FAILED(hr))
   {
          return hr;
   }

   hr = pSample->GetPointer(&pData);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }

   hr = GetPixelData(hBmp, &pData, &iSize);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }

   hr = pSample->SetActualDataLength(iSize);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }

   hr = pSample->SetSyncPoint(TRUE);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }
   
   hr = this->Deliver(pSample);
   pSample->Release();

   return hr;
}

GetMediaSample allocates a sample from the memory allocator in the base CBaseOutputPin class and passes the start and end times which can also be zero:

HRESULT CLiveSourceStream::GetMediaSample(IMediaSample** ppSample)
{
   REFERENCE_TIME rtStart = m_lastFrame;
   m_lastFrame += m_rtFrameRate;

   return this->GetDeliveryBuffer(ppSample, &rtStart, &m_lastFrame, 0);
}

GetPixelData validates the bitmap parameters against the previously set (or default) parameters, and if they are OK, returns the pointer to the pixel data and the data size:

HRESULT CLiveSourceStream::GetPixelData(HBITMAP hBmp, BYTE** ppData, int* pSize)
{
   ASSERT(hBmp);
   
   BITMAP bmp = {0};
   int res = ::GetObject(hBmp, sizeof(BITMAP), &bmp);
   if(res != sizeof(BITMAP))
   {
          return E_FAIL;
   }

   if(bmp.bmBitsPixel != m_bmpInfo.biBitCount ||
      bmp.bmHeight != m_bmpInfo.biHeight ||
      bmp.bmWidth != m_bmpInfo.biWidth)
   {
          return E_INVALIDARG;
   }

   *pSize = bmp.bmWidthBytes * bmp.bmHeight;
   memcpy(*ppData, bmp.bmBits, *pSize);

   return S_OK;
}

After successfully delivering the media sample, you need to release it as it is a simple COM object and uses reference counting for its memory management. Note that downstream filters that need the sample beyond the Deliver method call scope need to call AddRef on that sample.

COM plumbing

As you already know, filters are COM containers, which means you have to implement three basic COM constructs:

1. Static factory method for creating filter classes:

   CUnknown* WINAPI CLiveSource::CreateInstance(LPUNKNOWN pUnk, HRESULT *phr)
   {
      CUnknown* pNewFilter = new CLiveSource(pUnk, phr);
   
      if (phr)
      {
         if (pNewFilter == NULL) 
           *phr = E_OUTOFMEMORY;
         else
           *phr = S_OK;
      }
   
      return pNewFilter;
   }

2. When your filter implements some interfaces, you should override the QueryInterface method and increment the reference count by calling the GetInterface method:

   STDMETHODIMP CLiveSource::NonDelegatingQueryInterface(REFIID riid, void **ppv)
   {
      CheckPointer(ppv, E_POINTER);
   
      if(riid == IID_ILiveSource) 
      {
         return GetInterface((ILiveSource*) m_pOutputPin, ppv);
      } 
      else if(riid == IID_IAMPushSource)
      {
         return GetInterface((IAMPushSource*) m_pOutputPin, ppv);
      }
      else if(riid == IID_IAMFilterMiscFlags)
      {
         return GetInterface((IAMFilterMiscFlags*) this, ppv);
      }
      else 
      {
         return CBaseFilter::NonDelegatingQueryInterface(riid, ppv);
      }
   }

3. When implementing an interface like ILiveSource in this sample code, you should implement IUnknown by using the DECLARE_IUNKNOWN macro.

Using the code

After successfully building and registering the filter, you can use it in your code:

::CoInitialize(NULL);

graph.CoCreateInstance(CLSID_FilterGraph);
graph->QueryInterface(IID_IMediaControl, (void**)&media);
graph->QueryInterface(IID_IFilterGraph2, (void**)&m_filterGraph2);

pSource.CoCreateInstance(CLSID_CLiveSource);
pSource->QueryInterface(IID_ILiveSource, (void**)&pLiveSource);

m_filterGraph2->AddFilter(pSource, LIVE_FILTER_NAME);

IPin* outpin = NULL;
pSource->FindPin(LIVE_OUTPIN_NAME, &outpin);
m_filterGraph2->Render(outpin);
outpin->Release();

RECT area;
m_videoWnd.GetClientRect(&area);

CComPtr<IVideoWindow> pWnd = NULL;
graph->QueryInterface(IID_IVideoWindow, (void**)&pWnd);
pWnd->put_Owner((OAHWND)m_videoWnd.m_hWnd);
pWnd->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS);     
pWnd->SetWindowPosition(area.left, area.top, 
         area.right - area.left, area.bottom - area.top);
pWnd.Release();

media->Run();

And at some other place and may be in another thread, as the filter is thread safe, you can call AddFrame:

HBITMAP hBmp = GetFrame();
HRESULT hr = pLiveSource->AddFrame(hBmp);
if(FAILED(hr))
{
   // Handle error here !
}

::DeleteObject(hBmp);

References

1. Programming DirectShow for Digital Video and TV
2. Filter Base Classes
3. Live Source Filters

History

• 15.2.2011

Initial version.

• 13.3.2011

Fixed calling convention in the IAddFrame interface.