Most current code on github
Version : 0.9.0.9
Old version: 0.9.0.5
Introduction
Do you want to load a point cloud with little overhead, and a few lines of code?
Then do some mathematical operations on it like feature extraction, rotation, or more advanced: stitch point clouds together ?
Then you should read further :)
Background
I use the Kinect v2 to extract point clouds, and needed a simple code to display and handle point clouds. For this I did not find a simple utility in the internet.
I tried to use the Point Cloud Library, but a simple interface for .NET is missing. Using it in C++ is not quite easy: Compiling means that you include about 1 million header files. Compilation is slow and there are a lot of things to take care, a lot of group entries to read until you get it running on your particular system. Perhaps it is easier with Linux, but at least with Windows 64 bit you spend some time to get it running. This overhead is large, if you want to simply load a point cloud, and do some operations on it.
The VTK library seemed to be an alternative, but unfortunately the community is not that active any more, so it is seldom updated and the current version is quite old. I gave it a try however, see the link of my Kinect article. Using the VTK in .NET is not very nice since the C++ implementation is not well encapsulated in the C# port (Activiz), so that for instance you are not able to check values in the debugger very well, or need to take care of destroying objects yourself. Things that slow down your development.
These are only the top two of my trials.
For the first version published (see version 0.9.0.5):
I found a very nice C# program which does almost everything I wanted, from Douglas Andrade at cmsoft. He gave me the permission to take his code and extract a part of it to use it as a user control.
This is what I did, made some improvements here and there, and here is a nice small library which you can use in your project. The utility uses OpenTK which encapsulates and extends OpenGL.
Now that I have extracted this user control, perhaps it is useful also for other people doing point clouds handling. I think, many people engaged in scientific 3D investigations are interested in having a simple library.
The disadvantage might be that it is currently relying on a MS Windows operating systems, but it could be ported also to Mono.
The great advantage is that you can start right away using the code, and do not need days of work for finding dependencies or other overhead which you have to do with most of the other existing project.
Version 0.9.0.9
However, the performance was not good for large numbers of points, especially for real time scannning.
After a complete rework, almost nothing of the original code left, instead I used the shader technologs of OpenGL for a fast control
The code is work in progress, in particular you will find a lot of experimental calls for point cloud registration and alignment.
In some cases, the program will crash due to OpenGL errors - I did not debug all crah reasons.
If you want to help, please investigate and try to solve :)
Code cleaning is also highly necessary, a lot of unused code is still present - the old,unused code of the version 0.9.0.5 is not fully cleaned from the current version.
I have duplicate classes for PointClouds, also due to the not finished conversion.
Improving all that is more than I can do for an open source project, if someone wants to help, I will port all to Github, so that contributions can be made.
Prerequisites
Windows 7 or higher 64 bit
Graphic adapter with OpenGL driver 4.0 or higher
Microsoft .NET 4.6
Microsoft Visual Studio 2013 or higher - for the source code
Features of the user control
For a brief description of the features please follow the steps
1. Start the test program "OpenTKTest"
2. Load a point cloud e.g. the bunny.obj file, this is available in the bin/Model/UnitTests folder of the source and exe distribution. (The bunny.obf file is a freely available point cloud from the Stanford university, used in a lot of projects in the web: Link).
3. You may rotate the point cloud using the right mouse button, pan using left mouse button and zoom using the scroll button.
Change color of the model or background, etc. using the Tools-Settings dialog.
After you change the "View Mode" in the combo box on the top left hand side to "Triangles", you see something like this:
4. If you load more than one model, you can move one model in comparison to the other in space.
This is done by
-first loading two models, like load the bunny two times.
-in the combo box "Camera/Model" change to "Model"
-in the combo box "Selected Model" change to the first model
-now you can rotate / translate/scale the first model in comparison to the other
Please check out further usage yourself, like changing the size of the points etc., it should be straightforward.
Coding
The user control "OpenGLControl" is embedded in a Windows form of the test application.
You may simply add it in a windows form of your own application e.g. in the designer
It contains the GLControl from the OpenTK library, and is extended with mouse handling for rotate, translate, loading files etc. The instantiation of the looks like in the code:
this.glControl = new OpenTKLib.OGLControl();
The user control contains the OpenGLContext class, which contains all 3D data used in the control.
public class OpenGLContext
{
public Camera Camera;
The most important member is the List of 3D models, contained in the variable List<RenderableObject> RenderableObjects
One Renderable Object contains a PointCloud instance, which, as you can assume, contains the data for drawing.
For instance to load an object file, you would use the method
this.PointCloud = PointCloud.FromObjFile(fileName);
Rendering in OpenGL
The control uses shaders, which is a highly fragile construct.
Who programs with OpenGL has the feeling of going some 20 years in the past, somewhere between basic C and Assembler.
The basic shaders were introduced already in OpenGL 2.0 in 2004, but programmers still do not use them on large scale, most of the code samples are without,
Programming the shaders is even a bit more antiquary as is OpenGL, I wonder, why noone introduces object oriented programming in OpenGL. The lack of this caused a lot of trials to find alternatives to OpenGL, such as DirectX. But Microsoft did not succeed, mostly because OpenGL is platform independent, so we will will have to live with OpenGL for still a long time.
Shaders give a much better performance, so it is worth doing the work.
OK, so I adapted some open source code to have object orientation in the shaders.
The object displayed in OpenGL is :
RenderableObject which contains the PointCloud mentioned before and a variable for the shader.
A simple class diagram showing the classes from User control to the object that is rendered is:
The renderable object contains the Point cloud (a list of Vectors together with color informations, indices...), and a shader, necessary for the OpenGL drawing.
public abstract class RenderableObject
{
public PointCloud PointCloud = new PointCloud();
public Shader shader = new Shader();
While initializing the drawing, the method Shader.InitializeShaders() is called, which loads the shader file (vertex and fragment shader), then links them, sets the attributes and generates the Buffers.
public bool InitializeShaders(string filename_vshader, string filename_fshader, string path)
{
try
{
Init();
LoadShaderFromFile(path, filename_vshader, ShaderType.VertexShader);
LoadShaderFromFile(path, filename_fshader, ShaderType.FragmentShader);
Link();
ValidateLink();
SetAttributes();
SetUniforms();
GenBuffers();
}
On drawing the RenderableObject, the method RenderableObject.Render is called, which assigns the modelView matrix to the shader uniform address.
This makes drawing a bit more straightforward than before having shaders.
public void Render(PrimitiveType myRenderMode, PolygonMode myPolygonMode)
{
RefreshRenderableData();
this.shader.Use();
GL.UniformMatrix4(shader.GetUniformAddress("MVP"), false, ref this.mvp);
switch (myRenderMode)
{
case PrimitiveType.Triangles:
{
GL.PolygonMode(MaterialFace.FrontAndBack, PolygonMode.Line);
GL.DrawElements(PrimitiveType.Triangles, this.PointCloud.Indices.Length, DrawElementsType.UnsignedInt, 0);
break;
}
Fast rendering
If you want to use live streming to the control (e.g. during scanning), you might use the code:
this.openGLControl.GLrender.ReplaceRenderableObject(pcr, false);
This way you get an update rate of 10-20 frames per second when using a point could of about 100.000 points, on a computer with "medium" type hardware (about 800 EUR computer in 2016, all in all).
Some code samples for point cloud manipulations
1. Create a cube and show it in the window
public void ShowCube()
{
PointCloudVertices cube = PointCloudVertices.CreateCube_Corners(50);
OpenTKForm fOTK = new OpenTKForm();
fOTK.ShowPointCloud(cube);
fOTK.ShowDialog();
}
2. Translate a cube
public void TranslateCube()
{
PointCloudVertices = PointCloudVertices.CreateCube_Corners(50);
PointCloudVertices.Translate(verticesTransformed , 10, 3, 8);
}
3. Rotate a point cloud
PointCloudVertices.RotateDegrees(cube, 25, 10, 25);
4. Scale a point cloud
Vector3d vScale = new Vector3d(1, 1, 1000);
PointCloudVertices.ScaleByVector(cube, vScale);
5. The bunny file in various views
Source Code used
Various source code is used, see list
Points of Interest
OpenGL and OpenTK are very powerful tools, the math library of OpenTK is nice. When using it for more difficult matrix operations - which you need for image feature extraction or point cloud stitching - however, some extensions are needed, or other math libraries. e.g. for registration a singular value decomposition. needed for Iterative closest point algorithms.
History
Version 0.9.0.4 on 11/7/2014
Version 0.9.0.5 on 1/14/2015
-better rotation, translation, scale methods
-lots of new testcases
-added triangulation methods (w.i.p.)
Version 0.9.0.9 on 8/14/2016
-complete rework: control is now based on shaders
