Introduction
Silverlight Media Framework (or SMF), now a part of Microsoft Media Platform: Player Framework, is a feature-rich framework for developing media players in Silverlight. It contains a fairly comprehensive and skinnable UI, and supports a reasonable amount of video formats and delivery methods out-of-the-box. But currently, it's really lacking in the area of subtitles/closed-captions. This article will walk you through the creation of custom caption plugins to support any format you like.
The Code: Preamble
- At the time of writing, MMPPF version 2.5 is the latest 'stable'. All development has been done against this version.
- As this is extracted from a large scale closed-source commercial project, I cannot just provide the whole source, due to company rules, etc... Sorry about that!
- I will start by creating a base class, which isn't very useful if you only want to support one format, but I swear this is not due to over-engineering! Once again, this is an excerpt from a much larger codebase, which features more than one supported format.
- The file downloading part will be left out of the scope of this article. In my case, I have a complex utility class that handles both downloading and local
IsolatedStorage
caching. You can choose to handle the downloading in the plugin or outside, or use local files, whatever suits you best. A simplistic download solution is provided here.
- This has been moderately tested only, as the code is quite 'fresh', so some bugs can still be present. Please report them in the comments!
SMF Plugin Basics
Before writing SMF plugins, you can read some official documentation:
- The 'Plugin Developer Guide' is useful, it gives an overview and some details about the process of creating plugins, but it is not quite complete.
- The API documentation (CHM) comes in handy quite often, but is sometimes outdated (see the 'Architecture' page for an example: no
IMarkerProvider
!)
- The source code is sometimes the only real help, especially since it features some interesting samples, see '
Samples.MarkerProvider
' in this case.
As SMF plugins make use of MEF (Managed Extensibility Framework), it could be useful to get acquainted with this beast. This is not mandatory if you just want to get going, but it could help a lot when debugging plugin loading, concurrency or performance issues.
The Implementation
The Common Base Class: SubTitlePlugin
The IPlugin
interfaces contain quite a lot of members (see picture). Fortunately, you're probably using Visual Studio 2008 or 2010 to work with Silverlight, so you'll be able to save a lot of typing using the awesome 'implement interface' feature.
So let's get started writing code! First, I'll be creating a new empty class, named SubTitlePlugin
, abstract
and which implements Microsoft.SilverlightMediaFramework.Plugins.IMarkerProvider
. This class will provide all common features for subtitle plugins, reducing code 'noise' when working with actual plugins. This will be very much 'required boilerplate'. Anyway, if you're not copy/pasting my code, you will want to right-click IMarkerProvider
and select 'Implement interface' => 'Implement interface' in Visual Studio. This will generate stubs for all interface members, properties, methods and events. Now, let's fill our first public
method.
public void BeginRetrievingMarkers()
{
if (Source == null)
throw new ArgumentNullException("This requires a valid Source to download from");
_cancelRequested = false;
#if USE_DOWNLOAD_MANAGER
var localFileName = GetFileName(Source);
DownloadManager.DownloadFile(url = Source, localFileName = localFileName,
source = this, maxAge = null, completed = ReadSubtitles, error = DownloadError);
#elif USE_WEBCLIENT
var client = new WebClient();
client.DownloadStringCompleted += client_DownloadStringCompleted;
client.DownloadStringAsync(Source);
#endif
}
I've provided here a basic WebClient
-based retrieval that you might use, but in the next steps, I will use my custom DownloadManager
-based code. The DownloadManager
method parameters should be self-explanatory, anyway, the interesting point here is that once the file is successfully downloaded, it will call ReadSubtitles
, which will be our next method below, along with some private
members and the other callback method DownloadError
. If you were using the WebClient
method, the main difference is that you get a string
result instead of a byte[]
.
private bool _cancelRequested;
private MediaMarkerCollection<MediaMarker> _subtitles;
private void ReadSubtitles(object source, byte[] data)
{
if (_cancelRequested)
{
_cancelRequested = false;
return;
}
try
{
_subtitles = LoadMarkers(data);
}
catch (Exception ex)
{
if (RetrieveMarkersFailed != null)
RetrieveMarkersFailed(this, ex);
return;
}
if (NewMarkers != null)
NewMarkers(this, _subtitles);
}
private void DownloadError(object source, Exception error)
{
if (RetrieveMarkersFailed != null)
RetrieveMarkersFailed(this, error);
}
This is a very important method for our plugins. It calls the abstract protected
LoadMarkers
method, which will do the actual parsing of our chosen format. It also handles some mandatory event handling, especially the NewMarkers
call, where the plugin is notifying the SMFPlayer
that it has some content to provide, remember the interface name IMarkerProvider
? If you were to forget this call, nothing would happen at all for the end user! Your plugin could properly parse the plugins and all, but they would never show up in the player!
Note: If you don't like the endless if (x != null) x(...);
event firing constructs, the SMF source contains an extension method IfNotNull
, that you might want to check out.
Now we have the core of the code for SubTitlePlugin
! Most of the rest is automatic properties and events, except for the below two methods: Load
and Unload
. As their names imply, they contain initialization and cleanup code respectively.
public void Load()
{
try
{
DoLoadPlugin();
IsLoaded = true;
if (PluginLoaded != null)
PluginLoaded(this);
}
catch (Exception ex)
{
if (PluginLoadFailed != null)
PluginLoadFailed(this, ex);
}
}
public void Unload()
{
try
{
DoUnloadPlugin();
_subtitles = null;
IsLoaded = false;
if (PluginUnloaded != null)
PluginUnloaded(this);
}
catch (Exception ex)
{
if (PluginUnloadFailed != null)
PluginUnloadFailed(this, ex);
}
}
These methods are pretty straightforward: they handle event firing and provide two protected virtual
(hence optional) methods to allow derived class to perform specific loading/unloading tasks.
That's all for the base class. You can find it in the source package here.
A Concrete Sub-class: SamiProvider
Up to this point, we didn't have to use anything related to MEF. The base class isn't instanciable, so it doesn't require MEF decorators. Our concrete format-specific classes on the other hand require some decorator 'magic' to be found by the SMFPlayer
when it looks for plugins. The 'Plugin Developer Guide' provides good information here.
So, let's create a second class named SamiProvider
which will be derived from SubTitlePlugin
. Here is the whole code for this class:
[ExportMarkerProvider(
PluginName = PluginName,
PluginDescription = PluginDescription,
PluginVersion = PluginVersion,
SupportsPolling = false,
SupportedFormat = "SAMI")]
[PartCreationPolicy(CreationPolicy.NonShared)]
public class SamiProvider : SubTitlePlugin
{
#region MetaData
private const string PluginName = "SAMICaptionsPlugin";
private const string TypeAttribute = "type";
private const string SubTypeAttribute = "subtype";
private const string PluginDescription =
"Provides SMF with the ability to parse and display SAMI captions.";
private const string PluginVersion = "1.0";
#endregion
protected override MediaMarkerCollection<MediaMarker> LoadMarkers(byte[] data)
{
SamiLoader loader = new SamiLoader();
return loader.ParseSAMI(data);
}
}
Note: The whole MEF part is heavily based on the official sample mentioned earlier ('Samples.MarkerProvider
' in the sources for MMPPF).
The first decorator is ExportMarkerProvider
. Its three first parameters come from IPlugin
, they are just a textual identity for the plugin. The other parameters are specific to IMarkerProvider
.
SupportsPolling
is false
for this family of plugins, it's disabled in the SubTitlePlugin
(see PollingInterval
), it is not relevant for file-based, downloaded subtitles. It would be useful if using for instance a stream of markers.
- The last parameter
SupportedFormat
is actually the most important of all. It's this parameter that will allow selection of this plugin to handle SAMI subtitles.
The second decorator, PartCreationPolicy
, configures the instantiation of the plugin. I will direct you to the almighty MSDN for details. Here, we're saying we want one instance of the plugin per subtitle file. This is important as the retrieval and parsing are asynchronous, so we don't want multiple requests to mess up our plugin.
The SamiLoader
class is where I parse the downloaded data into a collection of MediaMarkers
. The real gotcha there is the use of CaptionRegion
. You should create one for each continuous block of text that you wish to display.
Using the Plugin
Now we have the structure to create plugins, but they won't be activated yet. In my project, I'm using a CustomPlaylistItem
, but for the sake of simplicity, I will just explain the method used. If you're interested by the custom playlist item, you can find it in the sources.
void SetupMarkers(PlaylistItem playListItem, string subtitleUrl)
{
string format = "SAMI";
var markers = new MarkerResource { Source = new Uri(subtitleUrl), Format = format };
if (playListItem.MarkerResources != null)
{
playListItem.MarkerResources.Clear();
}
else
{
playListItem.MarkerResources = new List<MarkerResource>();
}
playListItem.MarkerResources.Add(markers);
}
The key point here is the format
specification. This will allow the right plugin to be selected. If you have multiple plugins, you can pick the one that matches your subtitle format by passing the corresponding string
here.
History
- 20 Sept 2011 - First version