Vitruvius: Developing Kinect apps in minutes

Vangos Pterneas

0.00/5 (No votes)

26 Sep 2015CPOL7 min read

9.1K

During the past few months, I have been quite inactive. It was for a good reason, though. My team and I have been working on an ambitious project that helps Kinect developers build apps faster. I would like to introduce Vitruvius.

What is Vitruvius?

During the past few years, Kinect developers from all over the world have been creating outstanding Kinect apps, utilizing the power of Kinect SDK 2, Computer Vision, complex Mathematics, and Linear Algebra. Kinect development is hard and demanding. Through my blog posts (and your 600 comments), I have been trying to simplify the process and make it easier for students, researchers and engineers.

Kinect development should be fun.

So, after three years of professional experience and a lot of successful commercial projects, I developed a framework that accomplishes hard tasks using a few lines of code. Please welcome Vitruvius.

Vitruvius is a set of tools, utilities, controls and extensions that simplify Kinect development.

Features

Vitruvius is already used by big companies and institutions out there. I was proud to learn that NASA, XEROX and Michigan State University are actively using Vitruvius into their internal research projects.

Here’s exactly what’s included:

Avateering (3D model animation)

Animating a 3D model using Kinect is a very tough process. Not any more. Vitruvius lets you animate your FBX rigid models with one line of code! Never worry about the joint orientations or the complex Mathematics you need to apply. The following line of code animates a 3D model:

<code class="language-csharp">Avateering.Update(model, body);</code>

… and this is the result (supports up to 6 people concurrently):

Watch Avateering Video.

Vitruvius is bundled with 1 female and 1 male 3D model with 10 textures each. Load the FBX models, choose the skin, hair and clothes of your choice and go. Moreover, the samples include a virtual fitting room demo with 3 additional cloth models (a long-sleeve shirt, a pair of pants and a t-shirt) with 3 textures each. You are free to use them in your personal or commercial projects. The 3D models are customizable, so you can keep the internal Kinect skeleton and have your 3D artist modify the look and feel accordingly.

Kinect development is FUN with Vitruvius
Click To Tweet

Angle calculations

Calculating the angle between selected joints is also a matter of one line of C# code. Calculating an angle will help you take human body measurements with accuracy. You can specify the target axis (X, Y, Z) or simply calculate the angle in the 3D space. Vitruvius also provides you with a handy Arc control to display one or multiple angles on top of a body:

Watch video

The Arc control is available for both XAML and Unity.

HD Face Extensions

Kinect SDK includes the most powerful Face API in the world. Face and HD Face APIs lets you access over 1,000 facial points. Vitruvius provides an easy-to-use Face class that is populated with Eyes, Nose, Chin, Cheeks, Jaw and Forehead properties. No need to mess with the thousands of points. Accessing the facial properties has never been more straightforward:

<code class="language-csharp">var face = faceFrame.Face();
var forehead = face.Forehead;
var nose = face.Nose;
var jaw = face.Jaw;</code>

Watch video

Bitmap manipulation & Background removal

Converting a Kinect RGB, depth, or infrared frame into an image is, indeed, confusing. Bitmaps are handled by each programming platform. For example, XAML represents bitmaps using the WriteableBitmap class. Unity supports Texture2D. Vitruvius supports everything! We access the raw data of any Kinect frame and convert it to the corresponding image format.

<code class="language-csharp">var bitmap = colorFrame.ToBitmap();</code>

But people need more than just displaying an image. Using Vitruvius, you can accurately display the body on top of the Color, Depth, or Infrared frames, or even remove the background.

<code class="language-csharp">var bitmap = colorFrame.GreenScreen(depthFrame, bodyIndexFrame);</code>

Camera video – Background Removal video

Coordinate Mapping

Kinect developers have been struggling to convert points between the 3D world-space and the 2D screen-space. Kinect SDK includes CoordianteMapper, a useful tool that converts between camera space and depth/color space. Vitruvius simplifies Coordinate Mapping with the Vector and Point functions:

<code class="language-csharp">var joint = body.Joints[JointType.Head].Position;
var point = joint.ToPoint(Visualization.Color);</code>

Body Extensions

Handling body data effectively is a core part of my Kinect philosophy. A lot of people ask me how to identify the closest body, the number of the tracked joints, or the height of the players. The built-in Body class provides a lot of great information, but there is always something missing. Vitruvius fills these gaps and gives you everything you need to create a solid experience for your NUI applications:

<code class="language-csharp">// Get the closest body.
var body = frame.Bodies().Closest();

// Calculate the height in meters.
double height = body.Height();

// Find when someone entered or left the scene.
var controller = new PlayersController();
controller.PlayerEntered += Player_Entered;
controller.PlayerLeft += Player_Left;
</code>

Gesture detection

Natural User Interfaces are strongly relying on gestures. Gestures help users accomplish tasks easily using their hands or body. Vitruvius supports waving, swiping, and zooming.

<code class="language-csharp">void GestureRecognized(object sender, GestureEventArgs e)
{
   var gesture = e.GestureType;
}</code>

Documentation & Samples

Vitruvius supports WPF, Windows Store and Unity3D. There is extensive MSDN-like documentation for every platform.

The following demos are included in the Download section:

Avateering
Angles
Face
Fitting Room
Frame View
Gestures
Green Screen

Additionally, Vitruvius integrates amazingly well with your existing projects, too. No need to re-invent the wheel or change your coding style. Simply call any function you want from your existing code. It’s that simple!

Performance

We are taking performance seriously. Vitruvius is designed to use as few resources as possible. It only consumes what’s really necessary. There are no duplicates or unnecessary copies. Everything is real-time. As long as your computer meets the standard Kinect SDK requirements, you can use Vitruvius at no risk.

Who am I?

I guess that, since you are reading this blog post, you already know me of my Kinect tutorials, videos, and guides. I’m Vangos Pterneas, Microsoft Kinect Most Valuable Professional. I have been building commercial Kinect apps since 2012. Microsoft recognized my contributions to the open-source Kinect community and awarded me with the MVP title two years ago. My expertise includes C# and NUI app development. Vitruvius is crafted by a dedicated team of engineers and designers.

Ready? Download!

Vitruvius will empower your business or research and let you develop the next cutting-edge Kinect apps. Supports WPF, Windows Store, and Unity3D. NASA (yeap, the space guys), XEROX, and Michigan State University are already including Vitruvius in their workflow. It took me over 6 months to create this product. However, as a Microsoft MVP, I do not want to charge you a ton of money to access the cool features of Vitruvius. Kinect development should be accessible to everyone. This is why I am offering a free version that will let you try some of the aforementioned features at no cost.

Students, Professors and Researchers can benefit from the Academic version. The Academic version includes everything: avateering, angles, fitting room, bitmaps, gestures, HD Face. You can use it for research projects, thesis, and labs.
Small businesses and independent developers can benefit from Vitruvius Premium. The Premium version also includes everything, plus access to free updates and commercial support. Think about it: using the Premium version, you can spend a tiny amount of money and impress your customers with stunning applications!
Finally, bigger corporations can purchase Vitruvius Platinum and access all of the above benefits, plus dedicated telephone support. I am also offering an 1-on-1 consulting session (valued $280) that will help you get your business to the next level.