Detecting a Drone - OpenCV in .NET for Beginners (Emgu CV 3.2, Visual Studio 2017). Part 1

Introduction

Emgu CV is a .NET wrapper for OpenCV (Open Source Computer Vision Library) which is a collection of over 2500 algorithms focused on real-time image processing and machine learning. OpenCV lets you write software for:

• face detection
• object identification
• motion tracking
• image stitching
• stereo vision
• and much, much more...

Open CV is written in highly optimized C/C++, supports multi-core execution and heterogeneous execution platforms (CPU, GPU, DSP...) thanks to OpenCL. The project was launched in 1999 by Intel Research and is now actively developed by open source community members and contributors from companies like Google, Microsoft or Honda...

My experience with Emgu CV/OpenCV comes mostly from working on paintball turret project (which I use to have a break from "boring" banking stuff at work). I'm far from computer vision expert but I know enough to teach you how to detect a mini quadcopter flying in a room:

In the upper-left corner, you can see frame captured from video file, following that is the background frame (static background and camera makes our task simpler)... Next to it are various stages of image processing run before drone (contour) detection is executed. The last frame shows the original frame with drone position marked. Job done! Oh, and if you are wondering what is the "snow" seen in the video: these are some particles I put to make the video a bit more "noisy"...

I assume that you know a bit about C# programming, but are completely new to Emgu CV/OpenCV.

By the end of this tutorial, you will know how to:

• use Emgu CV 3.2 in C# 7 (Visual Studio 2017) application (most tutorials available online are quite outdated!)
• capture frames from video
• find changes between images (diff and binary threshold)
• remove noise with erode and dilate (morphological operations)
• detect contours
• draw and write on a frame

Sounds interesting? Read on!

The Code

I plan to give detailed description of the whole program (don't worry: it's just about 200 lines) but if you would like to jump straight to the code visit this GitHub repository: https://github.com/morzel85/blog-post-emgucv. It's a simple console app - I've put everything into Program.cs so you can't get lost!

Mind that because Emgu CV/OpenCV binaries are quite large these are not included in the repo. This should not be a problem because Visual Studio 2017 should be able to automatically download (restore) the packages...

Here you can download the video I've used for testing: http://morzel.net/download/emgu_cv_drone_test_video.mp4 (4.04 MB, MPEG4 H264 640x480 25fps).

STEP 0: Creating Project with Emgu CV

To start, let's use Visual Studio Community 2017 to create new console application:

Now we need to add Emgu CV to our project. The easiest way to do it is to use Nuget to install Emgu CV package published by Emgu Corporation. To do so, run "Install-Package Emgu.CV" command in Package Manager Console or utilize Visual Studio UI:

If all goes well, package.config and DLL references should look like this (you don't have to worry about ZedGraph):

Now, we are ready to test if OpenCV's magic is available to us through Emgu CV wrapper library. Let's do it by creating a super simple program that loads an image file and shows it in a window with obligatory "Hello World!" title:

using Emgu.CV; // Contains Mat and CvInvoke classes

class Program
{
    static void Main(string[] args)
    {
        Mat picture = new Mat(@"C:\Users\gby\Desktop\Krakow_BM.jpg"); // Pick some path on your disk!
        CvInvoke.Imshow("Hello World!", picture); // Open window with image
        CvInvoke.WaitKey(); // Render image and keep window opened until any key is pressed
    }
}

Run it and you should see a window with the image you've selected. Here's what I got - visit Kraków if you like my picture :)

The above code loads a picture from a file into Mat class instance. Mat is a n-dimensional dense array containing pointer to image matrix and a header describing this matrix. It supports a reference counting mechanism that saves memory if multiple image processing operations act on same data... Don't worry if it sounds a bit confusing. All you need to know now is that we can load images (from files, webcams, video frames, etc.) into Mat objects. If you are curious, read this nice description of Mat.

The other interesting thing you can see in the code is the CvInvoke class. You can use it to call OpenCV functions from your C# application without dealing with complexity of operating native code and data structures from managed code - Emgu the wrapper will do it for you through PInvoke mechanism.

Ok, so now, you have some idea about what Emgu CV/OpenCV libraries are and how to bring them into your application. Next post coming soon...

Update: Part 2 is ready!