Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / multimedia / video

EVR Presenter in pure C# with Direct3D Video Rendering

4.96/5 (15 votes)
12 Jul 2012CPOL6 min read 82.6K   4.1K  
Articles describes how to make pure C# rendering video on EVR with custom presenter over Direct3D in .NET

Image 1

Introduction

This is my second article of customizing video output with .NET and in pure C# code. More stuff is similar to my previous post so please review it, as I will not describe common stuff. This tutorial I think for advanced developers.

Before start

First you should read the article how to build EVR Presenter on MSDN especially prerequisites part.

Multimedia threading VS .NET threading

Yes, we should understand that stuff because .NET threading is totally different. MSDN description is good but not enough.

.NET threads and objects are all works and exist in specified execution context. Some objects are not available to be accessed from different threads for example form and controls. Other objects can be accessed from another thread but that require .NET to switch between execution contexts (there object created and there it is accessed). That operation can take a while or even hang the execution. Hanging can appear due different threading model. As in our case ALL multimedia threads are works in same context so you can access object created in other thread without any problem, only don’t forget about synchronization object. Hope I clear the difference and we can proceed.

Tracing, Debugging and Exception

I want to say some words regarding that before reviewing the code. Due different threading stuff I not suggest you to use the things like Trace.Write or Debug.Write in any code which processing multimedia data or accessing unmanaged resources frequently. This is also related to issue with switching threading context and as result degrade the performance. Raising exceptions are not recommended in the same issue, so better to checking returned values, using try catch statements also recommended. The way to solve tracing output is to use of OutputDebugString API.

C#
public static void TRACE(string _message) 
{ 
    if (!string.IsNullOrEmpty(_message)) _message += "\n"; 
    API.OutputDebugString(_message); 
}

Another helper function which will be useful along with above

C#
public static void TRACE_ENTER() 
{ 
    MethodBase _method = (new StackTrace(1,false)).GetFrame(0).GetMethod(); 
    TRACE(string.Format("{0}::{1}", _method.ReflectedType.Name, _method.Name)); 
}

This function print to an output window caller class name and method name.

Application Overview

Demo application shows how to perform video playback using DirectShow with Enhanced Video Renderer (EVR) with custom presenter. Presenter performing allocating the surfaces for playback, performing media type negotiations, synchronization of surfaces time stamps and display frames to the user using Direct3D9. Presenting surfaces done using SlimDX (managed library for Direct3D) similar as in my previous article.

Implementation scene presenting

Presenting the scene is similar to here.

Filter Graph

Filter graph a little different but it also particular graph for playback application, just it used the Enhanced Video Renderer instead of default and look like this:

Image 2

Class declaration and initialization

Same way I use my classes for graph building so the playback class declaration looks:

C#
public class DSFilePlaybackEVR : DSFilePlayback
    , IMFVideoDeviceID 
    , IMFVideoPresenter 
    , IMFGetService 
    , IMFTopologyServiceLookupClient

Here the inherited interfaces are required for the custom EVR presenter. We also have an event delegate and event variable in class to notify the scene that the surface is ready for display. The EVR filters declaration is:

C#
[Guid("FA10746C-9B63-4b6c-BC49-FC300EA5F256")] 
public class EVRRenderer : DSFilter 
{ 
    public EVRRenderer() 
        : base() 
    { 
    } 
}

EVR Class derived from base graph builder class which handles all basic stuff for playback via DirectShow we just need to override methods for initialization filters and connecting them:

C#
protected override HRESULT OnInitInterfaces() 
{ 
    m_evStop.Reset(); 
    HRESULT hr; 
    hr = (HRESULT)MFHelper.DXVA2CreateDirect3DDeviceManager9(out m_DeviceResetToken, out m_DeviceManager); 
    hr.Assert(); 
    if (hr.Succeeded) 
    { 
        hr = (HRESULT)m_DeviceManager.ResetDevice(Marshal.GetObjectForIUnknown(m_Device.ComPointer), m_DeviceResetToken); 
        hr.Assert(); 
    } 
    m_Caller = new Wrapper(this);  
    m_Renderer = new EVRRenderer(); 
    IMFVideoRenderer _renderer = (IMFVideoRenderer)m_Renderer.QueryInterface(typeof(IMFVideoRenderer)); 
    hr = (HRESULT)_renderer.InitializeRenderer(null, (IMFVideoPresenter)this); 
    hr.Assert(); 
    m_Renderer.FilterGraph = m_GraphBuilder; 
    hr = base.OnInitInterfaces(); 
    hr.Assert(); 
    return hr; 
}

Code fairly simple: we initialize DXVA2 device manager, create EVR filter and put it into the graph. After performing initialization EVR filter with our Presenter.

Implementing Presenter Interfaces

Now time for hardest part and you will know why I describe things regarding threading.

Invoker

Some methods of interfaces provided by my class for EVR Presenter is called from different threads, as if there will be one thread we will have no issues. But in there at least 2, commonly 3: user interaction (Play Pause Stop), media streaming (Samples delivering) and Clock (Samples Synchronization). Interfaces which are called at same context are IMFVideoDeviceID and IMFGetService. With other intarfaces we should do something to make it work properly, how to do so? The answer is simple: to make them to be called in same thread so context will be same. Hope you good enough with threading and synchronization to understand following code. Let’s look how implemented IMFTopologyServiceLookupClient:

C#
public int InitServicePointers(IntPtr pLookup)
{
    Wrapper.CCallbackHandler _handler =
                 new Wrapper.InitServicePointersHandler(m_Caller, pLookup);
    _handler.Invoke();
    return _handler.m_Result;
}

public int ReleaseServicePointers()
{
    Wrapper.CCallbackHandler _handler =
                 new Wrapper.ReleaseServicePointersHandler(m_Caller);
    _handler.Invoke();
    return _handler.m_Result;
}

Here we are simplify creates the specified pre-defined async invoker, wait for it result and return it. The invoker callback base class looks next:

C#
public class CCallbackHandler
{
    public bool m_bAsync = false;
    public CallType m_Type = CallType.Unknown;
    public EventWaitHandle m_Notify = new ManualResetEvent(false);
    public int m_Result = S_OK;
    private Wrapper m_Invoker = null;

    #region Constructor

    public CCallbackHandler(Wrapper _Invoker)
    {
         m_Invoker = _Invoker;
    }

    #endregion

    #region Methods

    public void Invoke()
    {
         m_Invoker.InvokeThread(this);
         WaitHandle.WaitAny(new WaitHandle[] { this.m_Notify, m_Invoker.m_Quit });
    }

    #endregion
}

And the actual invokers thread methods:

C#
private void InvokeThread(object _param)
{
	lock (m_LockThread)
	{
         		m_Parameter = _param;
                  m_Notify.Set();
	}
	WaitHandle.WaitAny(new WaitHandle[] { m_Quit, m_Ready });
}

private void ThreadProc(object _state)
{
    while (true)
    {
                 int nWait = WaitHandle.WaitAny(new WaitHandle[] { m_Quit, m_Notify });
                 if (nWait == 1)
                 {
                         object _param;
                         lock (m_LockThread)
                         {
                                _param = m_Parameter;
                         }
                         m_Ready.Set();
                         AsyncInvokerProc(_param);
                 }
                 else
                 {
                          break;
                 }
    }
}

We put the caller object as parameter after set the notify event which wakes up the thread to process the parameter and waits until the parameter will be retrieved. The thread got the parameter and executes passed callback; so all access to managed resources become from single thread.

Advanced Marshaling

Hope you still here; as previous part was not last hard code. Once the method is called with the invoker in same thread we can without any problems hold COM interfaces in our class. But someone who try to make EVR Presenter in .NET I’m sure had an issue with the InitServicePointers method of IMFTopologyServiceLookupClient interface, right? The issue appear, as I remember (I wrote that code year or two ago) was with query IMFTopologyServiceLookUp from pLookUp. Issue happened because of COM have aggregation and the returned interface may not be the interface of an actual object and the .NET doesn’t handle that it just call the QueryInterface and fail if it not in there, but we are know it is here. To solve this we just can access the vtable (table of virtual methods) of interface we interested in. All entry in that table are the pointers to the functions in order of interface inheritance and interface methods (actually interface it is structure with the specified methods entries and nothing else). As an example first 3 entries in interface are always implementation of IUnknown in order QueryInterface, AddRef and Release, That is true for all managed interfaces too, plus also for managed objects, as all managed objects are COM objects by default (that just hidden from developers). So I made the helper class which allows accessing COM object by it vtable:

C#
public class VTableInterface : COMHelper,IDisposable
{
    #region Delegates

    private delegate int QueryInterfaceProc(
        IntPtr pUnk,
        ref Guid riid,
        out IntPtr ppvObject
        );

    #endregion

    #region Variables

    protected IntPtr m_pUnknown = IntPtr.Zero;

    #endregion

    #region Constructor

    protected VTableInterface(IntPtr pUnknown)
    {
        if (pUnknown != IntPtr.Zero)
        {
            m_pUnknown = pUnknown;
            Marshal.AddRef(m_pUnknown);
        }
    }
    ~VTableInterface()
    {
        Dispose();
    }

    #endregion

    #region Methods

    public int QueryInterface(ref Guid riid, out IntPtr ppvObject)
    {
        ppvObject = IntPtr.Zero;
        if (m_pUnknown == IntPtr.Zero) return E_NOINTERFACE;
        QueryInterfaceProc _Proc = GetProcDelegate<QueryInterfaceProc>(0);
        if (_Proc == null) return E_UNEXPECTED;
        return (HRESULT)_Proc(m_pUnknown,ref riid,out ppvObject);
    }
    #endregion

    #region Helper Methods

    protected T GetProcDelegate<T>(int nIndex) where T : class
    {
        IntPtr pVtable = Marshal.ReadIntPtr(m_pUnknown);
        IntPtr pFunc = Marshal.ReadIntPtr(pVtable, nIndex * IntPtr.Size);
        return (Marshal.GetDelegateForFunctionPointer(pFunc, typeof(T))) as T;
    }

    #endregion

    #region IDisposable Members

    public void Dispose()
    {
        if (m_pUnknown != IntPtr.Zero)
        {
            Marshal.Release(m_pUnknown);
            m_pUnknown = IntPtr.Zero;
        }
    }
    #endregion
}

The main interesting method here is GetProcDelegate which allows getting method from vtable by it index. How it works you can see in QueryInterface implementation. So to implement IMFTopologyServiceLookUp without any problems we make next code:

C#
public class MFTopologyServiceLookup : VTableInterface, IMFTopologyServiceLookup
…
private delegate int LookupServiceProc(
            IntPtr pUnk,
            MFServiceLookUpType Type,
            uint dwIndex,
            [In, MarshalAs(UnmanagedType.LPStruct)] Guid guidService,
            [In, MarshalAs(UnmanagedType.LPStruct)] Guid riid,
            [Out, MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.SysInt)] IntPtr[] ppvObjects,
            [In, Out] ref uint pnObjects);
….
public int LookupService(MFServiceLookUpType _type, uint dwIndex, Guid guidService, Guid riid, IntPtr[] ppvObjects, ref uint pnObjects)
{
            if (m_pUnknown == IntPtr.Zero) return E_NOINTERFACE;
            LookupServiceProc _lookUpProc = GetProcDelegate<LookupServiceProc>(3);
            if (_lookUpProc == null) return E_UNEXPECTED;
            return (HRESULT)_lookUpProc(
                        m_pUnknown,
                        _type,
                        dwIndex,
                        guidService,
                        riid,
                        ppvObjects,
                        ref pnObjects
                        );
}

Not so hard I think. Forgot to mention the interface delegate function differ from method declaration in interface in one additional argument. First argument should be the pointer to the vtable object or our IntPtr, why that necessary you can find over web I think.

Samples Scheduler and notify of free sample

If you look at the EVR Presenter C++ example from Microsoft you can find that it define free samples while it released, I mean called Release with specified notification set on that. .NET doesn’t allow us to use that method as we have no access to the IUnknown directly. So I solve that with usage of events (probably for someone better to use semaphores, but this is just an example).

Main Application

Implementation of main form is very easy, most interesting methods I describe here. Variable declaration for scene and playback:

C#
private Scene m_Scene = null; 
private DSFilePlayback m_Playback = null;

Creating scene object:

C#
m_Scene = new Scene(this.pbView);

Here pbView control on which we’ll do presenting the video. Starting playback code:

C#
m_Playback = new DSFilePlaybackEVR(m_Scene.Direct3DDevice); 
m_Playback.OnPlaybackStop += new EventHandler(btnStart_Click); 
((DSFilePlaybackEVR)m_Playback).OnSurfaceReady += new EVR.SurfaceReadyHandler(m_Scene.OnSurfaceReady); 
m_Playback.FileName = this.tbFileName.Text; 
if (m_Playback.Start().Succeeded) 
{ 
    btnStart.Text = "Stop"; 
    btnBrowse.Enabled = false; 
}

In this code we create EVR rendering graph with specified Scene device. After we provide event handler for surface ready notify, and starting playback. Stopping code is fairly simple – just Dispose the playback:

C#
m_Playback.Dispose(); 
m_Playback = null;

History

Initial Version 11-07-2012

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)