Using third-party filters in a video application in C#

• Download executables files - 222 Kb
• Download source code - 10 Kb

Introduction

DirectShow filters are the basic building blocks of multimedia applications on the Windows platform. Many of these filters are provided by Microsoft as part of their operating systems. But even more filters are created by ISVs and developers. In this article, we'll see how to use filters that we have written previously in a simple video application.

Background

In a previous article, I created two filters in C#. In this article, I'll create a simple application that makes use of these filters. The application will have two windows: one that displays the original video and, the second one showing the result of applying the filters on the video. Moreover, the application has widgets to control the video and the threshold value of the Sobel transformation.

Besides testing the filters, we'll make use of the "Infinite Tee" filter from MS in order to duplicate the video stream for our two windows. The graph created for this application is not completely trivial, and we'll have the opportunity to write a couple of methods in order to simplify the task.

Using the code

In order to run the application, the Black and White filter and the Sobel transformation filter must be installed on your machine. More information can be found in my previous article. In order to write programs that make use of these filters, or any other third-party filters, you need to have the GUID of the filter. They should be provided by the supplier; in our case, we have the source code of the filters and we find that the GUIDs, we are looking for, are:

// the Guids for our custom filters

Guid sobelGuid = new Guid( "1C826B9A-4008-4C41-B601-A783A40AFAB2" );
Guid bwGuid = new Guid( "0C017086-684E-41c9-A4BB-640570C64B28" );

Moreover, our Sobel filter provides a custom COM interface, so our code has the following declarations:

// 

// The ISobel interface is a custom COM interface

// exposed by the Sobel filter, in order to access it

// from C#, we need to define it

//

[Guid("263AFD9E-465C-4dde-9BB1-168C25E2B87F"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface ISobel
{
    int SetThreshold( int newValue );
}

Points of Interest

Besides some common DirectShow interfaces, we use the following objects:

// We'll use an Infinite Tee filter object to duplicate the

// video stream by two

InfTee tee = null;

// We'll only treat wmv format 

WMAsfReader reader =  null;

// The Sobel object and filter interface

object sobelObject = null;
IBaseFilter sobel = null;
// the black and white filter interface

IBaseFilter bw = null;

The Infinite Tee filter is a MS-supplied filter that delivers the sample received on its input pin to an arbitrary number of output pins. We use it to have one sample (or in our case, video frame) displayed without any transformation and a copy of the frame fed to our black and white filter.

The graph we'd like to build is the following:

We have to add the objects for the Black and White, the Infinite Tee, and Sobel filters to the filter graph. There is nothing unusual about these steps. Then, we configure a WMAsfReader object (if you'd like to treat other file types besides those from Windows Media, you'd have some extra code to write here). Now, we have to connect the pins of the Infinite Tee filter. For this, we have the following code:

// connect video source to infinite tee filter

IPin teeInput = DsFindPin.ByDirection( (IBaseFilter)tee, 
                                PinDirection.Input, 0 );
IPin vidOutput = DsFindPin.ByDirection( (IBaseFilter)reader, 
                                   PinDirection.Output, 0 );
// check if the video stream is on the first pin

if( CheckVideo( vidOutput ) )
  graphBuilder.Connect( vidOutput, teeInput );
else // video stream is on second pin

{
  vidOutput = DsFindPin.ByDirection( (IBaseFilter)reader, PinDirection.Output, 1 );
  graphBuilder.Connect( vidOutput, teeInput );
}

First, we grab the input pin of the Infinite Tee filter and the output pin of the WMAsfReader, and we check if the first output pin of the WMAsfReader is for a video stream. If it is, we connect this pin with the Infinite Tee input pin. If it is not a video stream, we assume that the second pin is a video stream and we connect it to the Infinite Tee filter. Since the first output pin of the Infinite Tee filter is now a video stream, we call Render (not shown) to complete this part of the graph. Because the Black and White filter and the Sobel filter were added to the graph before this call, "Intelligent Connect" will insert these before the video renderer that it will add to the graph to render this pin. Then, we configure the first video renderer window, in particular, we set its owner property to one of the panels used by our form. We expect this condition to be satisfied later on when we search the graph for the second video renderer.

Then we call Render for the second output pin of the Infinite Tee filter. Since our custom filters have been used in the first call to Render, this call will create a "standard" path to a new video renderer object. Now, we are faced with a problem: we have two video renderers in the graph and we would like to configure the second one, how do we do that? Well, the method GetSecondRenderer was written to simplify this task:

//

// This method find the second renderer in the filter graph;

// it assumes that the first renderer had its "owner" property set

//

IVideoWindow GetSecondRenderer() 
{
  IEnumFilters enumFilters;
  ArrayList filtersArray = new ArrayList();

  IFilterGraph filterGraph = (IFilterGraph)fg;
  filterGraph.EnumFilters(out enumFilters);
  
  IBaseFilter[] filters = new IBaseFilter[1];
  int fetched;

  while(enumFilters.Next(1, filters, out fetched) == 0)
  {
    IVideoWindow ivw = filters[0] as IVideoWindow;
    if( ivw != null ) 
    {
      IntPtr outPtr = new IntPtr();
      ivw.get_Owner( out outPtr );
      if( outPtr == IntPtr.Zero )
        return ivw;
    }
  }
  return null;
}

We enumerate all the filters in the graph and we QI for the IVideoWindow interface. If this succeeds, then we check the "owner" property. If it is "null", we know that we have found the correct renderer. After that, we set a few properties and we are ready to run the graph.

One interesting event handler is the handler for the trackbar ValueChanged event. We want to change the threshold for the Sobel transformation filter. But this property is a custom COM interface on the Sobel filter object, so we use the following code:

// access the ISobel interface on the sobel object

ISobel isobel = sobelObject as ISobel;
if( isobel != null ) 
{
  isobel.SetThreshold( trackBar1.Value );
  label1.Text = "Treshold value: " + trackBar1.Value.ToString();
}

We query the COM component object for the ISobel interface and we call the SetThreshold method on this interface.

Limitations and known issues

The Black and White filter is really minimal, its processing will work for common "320X240" video files, but might run into difficulties with other settings.

There is small lag between the original frame and the transformed one. As Sobel transformation is computationally expensive, this was to be expected but I haven't measured the "Infinite Tee" overhead (if any).

The application only treats Windows Media Files, it wouldn't be hard to modify it in order to treat .mpeg and .avi files, for examples.