Easy application to detect and recognize faces with EmugCV

• Download FacialRecognize.zip - 1.9 MB

Introduction

The purpose of this article is to demonstrate how easy is to make application using EmguCV (version 3.1) for detection and recognition faces in real time. This article assumes that the reader knows Entity Framework and must have a camera.

Background

The EmguCV is .NET wrapper to the OpenCV library, which allows you to use the respective functions in an .NET environment, in this case it's C#.

EmguCV uses the viola-Jones algorithm to solve the problem of detection faces in an effective and fast way. This algorithm compares existing contrasts in several areas (rectangular or square shape) in an image using techniques such as classifiers functions Haar cascade, integral image and AdaBoost learning algorithm. However, the innovation of this algorithm is the classifiers cascade (degenerative decision tree) that speeds up this process. 

First gather the tools 

• Download emguCV at http://www.emgu.com/wiki/index.php/Download_And_Installation

(Just to use the haar file)

• Create a new project;

(c# language and windows forms application) 

• Install emguCV using NuGet;

• Add the tools to the toolbox

(Over the toolbox tab click right mouse button and select "Choose Items". Then search for emgucv and add the controls to the toolbox)

Second structure your work

We have our tools ready, now lets structure our solution. First create a folder called  HaarCascade and then add the Haar file (haarcascade_frontalface_default.xml) to the folder. This file is in a folder called "bin" at the package you downloaded at the beginning of this article, this haar cascade it's just to detect faces. You can download others haar files that detect eyes, full body and also objects.

Third, let's get to work

To teach the system to recognize faces we need to save the training samples (pictures). In this application we will store the picture (sample) taken on a SQL database. Do to this create a local database and add Entity Data Model.

Add a new Entity named Faces with four scalar property named Id (Int32), Username (string), FaceSample (Binary) and UserID (Int32). This entity/table  will store the pictures samples to train the system.

Add a form to the project and also add three controls: imagebox (the new added control), textbox, button and a timer with 0.1 s interval, this timer it's to run the method to detect the user face.

Using the code

Form class code

The form class its where the business logic and UI its applied.

The logic its simple, detect a face and if the user takes a picture (tagged with the user name), the application will save the sample (picture) in the database and train the system to predict his face.

Do to this we must start loading the cascade classifier, setting the EigenFace method, and teach the system with pre-samples.

Quote: from wikipedia

Eigenfaces is the name given to a set of eigenvectors when they are used in the computer vision problem of human face recognition. 

private void start()
{
    cascadeClassifier = new CascadeClassifier(Application.StartupPath + @"\HaarCascade\haarcascade_frontalface_default.xml");
    capture = new Capture();
    //emugcv have others methods to reconize an object/face like FisherFaceRecognizer.
    //but i use the eigen because its flexible to noise
    faceRecognizer = new EigenFaceRecognizer(80, double.PositiveInfinity);
    learn();
    timer1.Enabled = true;
}

The learn method as the objective to train the system to recognize faces.

This method will return all the samples from the database streaming it from byte to bitmap and train the system with just two parameters (bitmap and userid).

/// <summary>
/// Teach the system with samples to predict the face
/// </summary>
/// <returns></returns>
public bool learn()
{
  List<Faces> allFaces = da.GetTrainingSample();
  if (allFaces.Count > 0)
   {
     var faceImages = new Image<Gray, byte>[allFaces.Count];
     var faceLabels = new int[allFaces.Count];
     for (int i = 0; i < allFaces.Count; i++)
         {
           Stream stream = new MemoryStream();

           stream.Write(allFaces[i].FaceSample, 0, allFaces[i].FaceSample.Length);

           var faceImage = new Image<Gray, byte>(new Bitmap(stream));

           faceImages[i] = faceImage;
           faceLabels[i] = allFaces[i].UserID;
          }

    faceRecognizer.Train(faceImages, faceLabels);                
   }
 return true;
}

FaceRecognize method it's for detect a face and put a square frame on it and also a username (or john doe if the face was not recognized) on the camera image.

private void faceRecognize()
{
  string name;
  //take a frame
  using (var imageFrame = capture.QueryFrame().ToImage<Bgr, byte>())
   {
     if (imageFrame != null)
       {
         //convert to gray to improve the prediction
         var grayFrame = imageFrame.Convert<Gray, byte>();
                    
         //Finds rectangular regions (face); the second param its the scale > 1 slowest process but accurated prediction
         var faces = cascadeClassifier.DetectMultiScale(grayFrame, 1.3, 6, new Size((grayFrame.Size.Width / 4), (grayFrame.Size.Height / 4)));

         //Predict the frame taked. If theres a sample of the face in db it will return the username, if not will say John Doe
         name = (facePredict(grayFrame) > 0) ? da.GetUsername(facePredict(grayFrame)) : "John Doe";

         foreach (var face in faces)
            {                        
              //draw a box at the face
              imageFrame.Draw(face, new Bgr(Color.Green), 2);
              //put text bellow the box
              CvInvoke.PutText(imageFrame, name, new Point(face.Location.X + 10, face.Location.Y - 10), Emgu.CV.CvEnum.FontFace.HersheyComplex, 1.0, new Bgr(0, 255, 0).MCvScalar);
            }
         imageBox1.Image = imageFrame;
       }
   }
}

Add a sample to the database when the user click the button.

The more samples from the same person the more accurate the system will be. Remember that lighting also influence the recognizing so its better to take a picture with different kind of lights and different daytime.

private void button1_Click(object sender, EventArgs e)
{
  userPicture = capture.QueryFrame().ToImage<Gray, byte>();

  //Finds rectangular regions (face)
  var faces = cascadeClassifier.DetectMultiScale(userPicture, 1.3, 6, new Size((userPicture.Size.Width / 4), (userPicture.Size.Height / 4)));

  foreach (var face in faces)
   {
     //resize sample
     faceImageThumb = userPicture.Copy(face).Resize(64, 64, Emgu.CV.CvEnum.Inter.Cubic);
   }

  //add to db and notify user
  MessageBox.Show(da.AddSample(textBox1.Text, ConvertImageToByte(faceImageThumb)), "Face", MessageBoxButtons.OK);
}

DataAccess class

This classes does the CRUD for the local SQL database.

The AddSample will search the entity faces for a samples of the user if exists will save the new sample and give the same user id (otherwise wil give a new user id).

  class DataAccess
    {
        /// <summary>
        /// add sample the more samples of the same person, the accurated the recognizing
        /// </summary>
        /// <param name="username"></param>
        /// <param name="faceBlob"></param>
        /// <returns></returns>
        public string AddSample(string username, byte[] faceBlob)
        {
            int userId = GetUserId(username);
            int rowChanged;
            Faces newFace = new Faces();
            try
            {
                using (RecModelContainer context = new RecModelContainer())
                {
                    newFace.FaceSample = faceBlob;
                    newFace.UserID = userId;
                    newFace.Username = username;
                    context.Faces.Add(newFace);
                    rowChanged = context.SaveChanges();

                    if (userId == 0)
                    {
                        newFace.UserID = newFace.Id;
                        context.SaveChanges();
                    }
                }
            }
            catch (DbUpdateException e)
            {
                return "Error " + e.InnerException;
            }
            return (rowChanged > 0) ? "Great a new face to recognize" : "Somehow, someway the face wasn't save.";
        }
    }

References

• http://www.emgu.com/wiki/index.php/Main_Page
• http://en.wikipedia.org/wiki/Viola–Jones_object_detection_framework
• https://en.wikipedia.org/wiki/Eigenface

History

06-09-2016 : Release