Introduction
I'm a newbie to image processing but not to programming. I uploaded this project as a demo code resource for other newbies who wish to get a quick start and feel on OpenCvSharp3 coding API. This is an article to show how C++ code is converted to OpenSharp3 code. You can experiment with the code parameters and look-up the C++ document for API's actions.
I considered the C# wrapper class emgu and OpenCV.NET... But I felt that OpenCvSharp3 was the simplest and better supported platform.
The demo code is sectioned off (as independent regions) to make grasping the code intent very simple. I had to compile the https://github.com/shimat/opencvsharp/releases sample code at least a couple of times to get a clean build.
Background
OpenVC is a popular C++ library for computer vision. It processes image pixels to find features of interest. However, C++ is an unmanaged code platform and a little awkward compared to C#. Because of this, I gave up C++ programming for C#.
Using the Code
To run the demo, create a new console app and copy the image and program files to it. Include one of the program files (program, program2 or program3) and exclude the other program files from the project build. Then uncomment (if applicable) one of the code regions and compile and execute that command selection. Change API parameters as desired and recompile and run. Note that you need to use the Visual Studio nuget manager plug-in tool to install the OpenCvSharp3
library. The basic package will include the OpenCv C++ DLLs.
using System;
using System.Collections.Generic;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.ML;
class Program
{
static void Main()
{
#region Canny Edge Detection
#endregion
#region Sobel Edge Detection
#endregion
#region Camcorder Capture
#endregion
#region Assigning pixel values
#endregion
#region Erode/Dilate Morphing
#endregion
#region draw a line
#endregion
#region draw a circle
#endregion
#region draw a Polygon
#endregion
#region draw a text string
#endregion
#region weighed filter
#endregion
#region find circles in image
#endregion
#region Get corners on image
#endregion
#region Machine Learning
int width = 512, height = 512;
Mat image = Mat.Zeros(height, width, MatType.CV_8UC3);
int[] labels = new int[] { 1, -1, -1, -1 };
float[,] trainingData = new float[,] { { 501, 10 }, { 255, 10 }, { 501, 255 }, { 10, 501 } };
Mat trainingDataMat = new Mat(4, 2, MatType.CV_32FC1, trainingData);
Mat labelsMat = new Mat(4, 1, MatType.CV_32SC1, labels);
SVM svm = SVM.Create();
svm.Type = SVM.Types.CSvc;
svm.KernelType = SVM.KernelTypes.Linear;
svm.TermCriteria = new TermCriteria(CriteriaType.MaxIter, 100, 1e-6);
svm.Train(trainingDataMat, SampleTypes.RowSample, labelsMat);
Vec3b green = new Vec3b(0, 255, 0), blue = new Vec3b(255, 0, 0);
for (int i = 0; i < image.Rows; ++i)
for (int j = 0; j < image.Cols; ++j)
{
Mat sampleMat = new Mat(1, 2, MatType.CV_32F, new float[] { j, i });
float response = svm.Predict(sampleMat);
if (response == 1)
image.Set<Vec3b>(i, j, green);
else if (response == -1)
image.Set<Vec3b>(i, j, blue);
}
int thickness = -1;
Cv2.Circle(image, new Point(501, 10), 5, Scalar.Black, thickness);
Cv2.Circle(image, new Point(255, 10), 5, Scalar.White, thickness);
Cv2.Circle(image, new Point(501, 255), 5, Scalar.White, thickness);
Cv2.Circle(image, new Point(10, 501), 5, Scalar.White, thickness);
thickness = 2;
Mat sv = svm.GetSupportVectors();
for (int i = 0; i < sv.Rows; ++i)
{
unsafe
{
float* v = (float*)sv.Ptr(i);
Cv2.Circle(image, new Point((int)v[0], (int)v[1]), 6, Scalar.Gray, thickness);
Console.WriteLine("{0:d}, {1:d}", (int)v[0], (int)v[1]);
}
}
Cv2.ImWrite("result.png", image);
Cv2.ImShow("SVM Simple Example", image);
Cv2.WaitKey(0);
#endregion
#region Feature SURF flann
#endregion
}
#region Support code for Get Corners on image region
static Mat getEdges(Mat image, int threshold)
{
Mat result = new Mat();
Cv2.MorphologyEx(image, result, MorphTypes.Gradient, new Mat());
applyThreshold(result, threshold);
return result;
}
static Mat createACross()
{
Mat cross = new Mat(5, 5, MatType.CV_8U, new Scalar(0));
for (int i = 0; i < 5; i++)
{
cross.Set<byte>(2, i, 1);
cross.Set<byte>(i, 2, 1);
}
return cross;
}
static Mat createADiamond()
{
Mat diamond = new Mat(5, 5, MatType.CV_8U, new Scalar(1));
diamond.Set<byte>(0, 0, 0);
diamond.Set<byte>(1, 0, 0);
diamond.Set<byte>(3, 0, 0);
diamond.Set<byte>(4, 0, 0);
diamond.Set<byte>(0, 1, 0);
diamond.Set<byte>(4, 1, 0);
diamond.Set<byte>(0, 3, 0);
diamond.Set<byte>(4, 3, 0);
diamond.Set<byte>(4, 4, 0);
diamond.Set<byte>(0, 4, 0);
diamond.Set<byte>(1, 4, 0);
diamond.Set<byte>(3, 4, 0);
return diamond;
}
static Mat createASquare()
{
Mat Square = new Mat(5, 5, MatType.CV_8U, new Scalar(1));
return Square;
}
static Mat createAXShape()
{
Mat x = new Mat(5, 5, MatType.CV_8U, new Scalar(0));
for (int i = 0; i < 5; i++)
{
x.Set<byte>(i, i, 1);
x.Set<byte>(4 - i, i, 1);
}
return x;
}
static void applyThreshold(Mat result, int threshold)
{
Cv2.Threshold(result, result, threshold, 255, ThresholdTypes.Binary);
}
static void IDTheCorners(Mat binary, Mat image)
{
for (int r = 0; r < binary.Rows; r++)
for (int c = 0; c < binary.Cols; c++)
if (binary.At<byte>(r, c) != 0)
Cv2.Circle(image, c, r, 5, new Scalar(255));
}
#endregion
#region nearest palette color averaging
static void binImage(Mat src, out Mat dst, int binSize = 51)
{
dst = src.Clone();
if (binSize <= 0 || binSize > 255) return;
for (int r = 0; r<src.Rows; r++)
for (int c = 0; c < src.Cols; c++)
{
Vec3b color = src.Get<Vec3b>(r, c);
int binCenter = binSize / 2;
dst.Set<Vec3b>(r, c, new Vec3b(
(byte)(((color[0] + binCenter) / binSize) * binSize),
(byte)(((color[1] + binCenter) / binSize) * binSize),
(byte)(((color[2] + binCenter) / binSize) * binSize)
));
}
}
#endregion
}
A Brief C++ to C# Walkthough
A full discussion on popular C++ API calls is available at opencvexamples.blogspot.com/p/learning-opencv-functions-step-by-step.html. The C# APIs are pretty much intuitive match to the C++ APIs. Also, see opencv home page for methods and classes discussion at: docs.opencv.org/3.1.0/ and (docs.opencv.org/2.4/ for backwards compatibility.)
The C# APIs namespace is at shimat.github.io/opencvsharp/, and github.com/shimat/opencvsharp/wiki.
The purpose of this walkthrough is to reflect the usage of the C# APIs when translating from C++ to C#. Note, we needed a CvTrackbar
reference... because the C# CvTrackbar
does not return an modified value within input arguments list... whereas the C++ counter API does return a modified value in the input parameter list. The intent of the walkthrough is not to explain the C++ library calls... but is to show the C# syntax required for translation.
using System;
using System.Collections.Generic;
using System.Windows.Forms;
using OpenCvSharp;
class Program
{
const int w = 500;
static CvTrackbar Track;
static Point[][] contours;
static HierarchyIndex[] hierarchy;
static void on_trackbar(int pos, object usedata)
{
int _levels = Track.Pos - 3;
Mat cnt_img = Mat.Zeros(w, w, MatType.CV_8UC3);
Cv2.DrawContours( cnt_img, contours, _levels <= 0 ? 3 : -1, Scalar.White,
2, LineTypes.AntiAlias, hierarchy, Math.Abs(_levels) );
Cv2.ImShow("contours", cnt_img);
}
static void Main()
{
Mat img = Mat.Zeros(w, w, MatType.CV_8UC1);
for( int i = 0; i < 6; i++ )
{
int dx = (i % 2) * 250 - 30;
int dy = (i/2)*150;
Scalar white = Scalar.White;
Scalar black = Scalar.Black;
if( i == 0 )
{
for( int j = 0; j <= 10; j++ )
{
double angle = (j + 5) * Math.PI/21;
Cv2.Line(img,
new Point(Math.Round(dx + 100 + j * 10 - 80 * Math.Cos(angle), 0),
Math.Round(dy + 100 - 90 * Math.Sin(angle), 0)),
new Point(Math.Round(dx + 100 + j * 10 - 30 * Math.Cos(angle), 0),
Math.Round(dy + 100 - 30 * Math.Sin(angle), 0)),
white, 1);
}
}
Cv2.Ellipse(img, new Point(dx + 150, dy + 100), new Size(100, 70), 0, 0, 360, white);
Cv2.Ellipse(img, new Point(dx + 115, dy + 70), new Size( 30, 20), 0, 0, 360, black);
Cv2.Ellipse(img, new Point(dx + 185, dy + 70), new Size( 30, 20), 0, 0, 360, black);
Cv2.Ellipse(img, new Point(dx + 115, dy + 70), new Size( 15, 15), 0, 0, 360, white);
Cv2.Ellipse(img, new Point(dx + 185, dy + 70), new Size( 15, 15), 0, 0, 360, white);
Cv2.Ellipse(img, new Point(dx + 115, dy + 70), new Size( 5, 5), 0, 0, 360, black);
Cv2.Ellipse(img, new Point(dx + 185, dy + 70), new Size( 5, 5), 0, 0, 360, black);
Cv2.Ellipse(img, new Point(dx + 150, dy + 100), new Size( 10, 5), 0, 0, 360, black);
Cv2.Ellipse(img, new Point(dx + 150, dy + 150), new Size( 40, 10), 0, 0, 360, black);
Cv2.Ellipse(img, new Point(dx + 27, dy + 100), new Size( 20, 35), 0, 0, 360, white);
Cv2.Ellipse(img, new Point(dx + 273, dy + 100), new Size( 20, 35), 0, 0, 360, white);
}
Cv2.ImShow( "image", img );
Point[][] contours0;
Cv2.FindContours( img, out contours0, out hierarchy,
RetrievalModes.Tree, ContourApproximationModes.ApproxSimple);
contours = new Point[contours0.Length][];
for( int k = 0; k < contours0.Length; k++ )
contours[k] = Cv2.ApproxPolyDP(contours0[k], 3, true);
Cv2.NamedWindow("contours", WindowMode.AutoSize);
Track = new CvTrackbar("levels+3", "contours", 4, 7, on_trackbar);
on_trackbar(0, 0);
Cv2.WaitKey();
}
}
Another interesting C++ to C# code translation I noted is the threshold demo at docs.opencv.org/2.4/doc/tutorials/imgproc/threshold/threshold.html. The Threshold_Demo
callback event required an alternative window definition specifying the window by name, causing a new thread or null
pointer to execute inside the C++ Library.
using OpenCvSharp;
class Program
{
static void Main()
{
TestCode t = new TestCode();
t.Function();
}
class TestCode
{
int max_BINARY_value = 255;
CvTrackbar CvTrackbarValue;
CvTrackbar CvTrackbarType;
Mat src_gray = new Mat();
Mat dst = new Mat();
Window MyWindow;
public void Function()
{
int threshold_value = 0;
int threshold_type = 3;
int max_value = 255;
int max_type = 4;
string trackbar_type = "Type: \n 0: Binary \n 1:
Binary Inverted \n 2: Truncate \n 3: To Zero \n 4: To Zero Inverted";
string trackbar_value = "Value";
Mat src = new Mat("../../Images/cartoon-train.png", ImreadModes.Color);
Cv2.CvtColor(src, src_gray, ColorConversionCodes.BGR2GRAY);
MyWindow = new Window("Track", WindowMode.AutoSize);
CvTrackbarType = MyWindow.CreateTrackbar(trackbar_type,
threshold_type, max_type, Threshold_Demo);
CvTrackbarValue = MyWindow.CreateTrackbar(trackbar_value,
threshold_value, max_value, Threshold_Demo);
Threshold_Demo(0, 0);
while (true)
{
int c;
c = Cv2.WaitKey(20);
if ((char)c == 27)
{ break; }
}
}
void Threshold_Demo(int pos, object userdata)
{
Cv2.Threshold(src_gray, dst, CvTrackbarValue.Pos,
max_BINARY_value, (ThresholdTypes)CvTrackbarType.Pos);
MyWindow.Image = dst;
}
}
}
Points of Interest
I rewrote some of the more common C++ implementation demo code to show how it works in OpenCvSharp. I find using the Vision Studio Object Browser to view the OpenCvSharp3
API method parameters very helpful. I also recommend viewing the https://github.com/shimat/opencvsharp/releases code samples too. It illustrates additional API calling code snippets and implementation.
History
I added this article because I found no information on OpenCvSharp3
at this website.
In the first update: I added 2 additional code snippets for completeness.
In the second update: I fixed the problem with the Draw FillPoly
code section.
In the third update: I only updated the article content. The downloadable code snippet section "nearest palette color averaging" works, but is slightly inferior to the code listed in the printout above. The download file doesn't allow for the bright colors. So, I added an offset value to the base colors. This makes the palette color closer to the real average and allow for pure white.
Also, because a nuget search on "opencvsharp
" only shows version 2, I updated the text to reflect "opencvsharp3
".
In the fourth update: I added 2 more complex examples to the code set.
In the fifth update: I added unsafe code to fix the float pointer translation in the Machine Learning snippet section.
In this update: I added a simple code walkthrough section.