Using FaceDetector in Windows 10 apps

Hey everyone! As I pointed out in a previous article, I think that the best way to learn something is to teach others, so that's what I’m doing now. I started to go through the new API's in Windows 10 and I decided to play a little with the FaceDetector. It took me almost a half an hour to get it to work because I wasn't able to find any samples, so I’ll share with you how I got it to work and what issues you may encounter. I haven’t used face detection until now so feel free to comment if I say or do anything stupid.

I’ll just show you something simple. The FaceDetector class has a method called DetectFacesAsync, which returns a list of detected faces in a SoftwareBitmap. Our task will be to get the number of faces in a picture. Here are the basic steps:

1. Get an image
2. Create a SoftwareBitmap from that image
3. Use the method DetectFacesAsync to get the list of faces in the SoftwareBitmap

I’ll try to keep things as simple as possible, so in order to get an image we’ll just load one from the web. I found a picture on Wikipedia with a guy that makes different faces and I think it’s perfect for our demo because we can see how many of those faces it detects.

Different faces of a man

The first step will be to download the image, and I’ll do this in the Loaded event of the page.

private static async Task DetectFaces()
{
    var path = "https://upload.wikimedia.org/wikipedia/commons/c/c4/Different_faces_of_a_man.jpg";
    HttpClient client = new HttpClient();
    var bytes = await client.GetByteArrayAsync(new Uri(path));
}

Now that we have our image, it’s time to create a SoftwareBitmap from it. This part was a little tricky but thanks to this post I found out that I need a BitmapDecoder to create the SoftwareBitmap. Here’s how you do this:

var stream = bytes.AsBuffer().AsStream();
var decoder = await BitmapDecoder.CreateAsync(BitmapDecoder.JpegDecoderId, stream.AsRandomAccessStream());
var softwareBitmap = await decoder.GetSoftwareBitmapAsync();

The decoder needs a random access stream, which we can get from the bytes of the image, and then we use the method GetSoftwareBitmapAsync. After we get our SoftwareBitmap it’s time to create an instance of the FaceDetector class. This class doesn’t have a public ctor, so we will use the static method FaceDetector.CreateAsync() to get an instance of it.

One other issue that I had with this was that I got exceptions when I was using the DetectFacesAsync() method. The reason was that FaceDetector supports different bitmap pixel formats from device to device, so the SoftwareBitmap that we provide has to have a pixel format supported by our FaceDetector. Fortunately we can see which formats it supports and convert our SoftwareBitmap to one of them. The FaceDetector class has a static method called FaceDetector.GetSupportedBitmapPixelFormats() which returns a list of supported formats. Here’s the full code of the method:

private static async Task DetectFaces()
{
    var path = "https://upload.wikimedia.org/wikipedia/commons/c/c4/Different_faces_of_a_man.jpg";
    HttpClient client = new HttpClient();
    var bytes = await client.GetByteArrayAsync(new Uri(path));
    var stream = bytes.AsBuffer().AsStream();

    var decoder = await BitmapDecoder.CreateAsync(BitmapDecoder.JpegDecoderId, stream.AsRandomAccessStream());
    var softwareBitmap = await decoder.GetSoftwareBitmapAsync();

    var detector = await Windows.Media.FaceAnalysis.FaceDetector.CreateAsync();
    var supportedBitmapPixelFormats = Windows.Media.FaceAnalysis.FaceDetector.GetSupportedBitmapPixelFormats();
    var convertedBitmap = SoftwareBitmap.Convert(softwareBitmap, supportedBitmapPixelFormats.First());

    var detectedFaces = await detector.DetectFacesAsync(convertedBitmap);
    await new MessageDialog("The image has " + detectedFaces.Count + " faces").ShowAsync();
}

You can see that I’m getting a list of supported bitmap pixel formats and I’m using the first one to get a converted SoftwareBitmap. My device apparently supports two formats called Nv12 and Gray8. I noticed that if I used the first one, the face detector finds 36 faces, but if I use the second one it finds only 33. I’ll say this again, it’s my first time using face detection and I don’t really understand why there is this difference, but I thought you should know about it. Finally, I use the DetectFacesAsync to get the list of detected faces and then I just show a MessageDialog with the number.

This is just a simple example to get you started with FaceDetector in Windows 10 apps. When I started writing the article I was unaware that Microsoft published code samples on Github and they also have a sample for basic face detection, so if you want to learn more you should check them out.

The post Using FaceDetector in Windows 10 apps appeared first on Bogdan Bujdea.