This article shows how to  construct OpenCV Image from data comming from PrimeSense Kinect drivers.

Problem

OpenCV and its C# wrapper are great tools in your SW toolkit that helps processing of collected data from Kinect. To start using its power, you first need to instantiate an OpenCV Image class from  Kinect Depth.

Solution

The code  takes depth map from _depthGenerator (OpenNI.DepthGenerator) and calculates gray image (new Image<Gray, Byte>(width, height)). It works in a situation where Kinect is placed above some solid desk.

• When first depth frame is processed, distances from Kinect are remembered in array TableDistancesYX. This should happen when no object is on the desk.
• When succeeding depth frames are processed (imagine a hand is placed on the table), whenever depth differs more than minimalFingerHeight, it is marked as gray in result depthImage.
• Code presented uses unsafe sequences and array pointers to maximize the speed (you need to set up compiler with unsafe option).

The result EmguCV image has only black or gray pixels:

Working C# code sample

This code is part of TouchTable project.

Note:  Sorry guys for the poor quality of code formatting, please copy paste it to Visual Studio and reformat it. Hopefully I will upgrade soon to new version of WordPress to get nicer code samples but I am waiting for the migration of my MySQL database to version 5.

private unsafe void processDepthFrame()
{
var depthMd = new DepthMetaData();
// process pDepth
_depthGenerator.GetMetaData(depthMd);
var pDepth = (ushort*)_depthGenerator.DepthMapPtr.ToPointer();
ushort* pCurrentDepth = pDepth;
var width = depthMd.XRes;
var height = depthMd.YRes;
depthImage = new Image<Gray, Byte>(width, height);
var data = depthImage.Data;
var minimalFingerHeight = Config.MinimalFingerHeight;
fixed (int* pEnvDistances = _envDistances) {
fixed (int* pTableDistances = TableDistancesYX) {
fixed (byte* pImageData = data) {
var pCurrEnvDistance = pEnvDistances;
var pCurrTableDistance = pTableDistances;
var pCurrImageData = pImageData;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++, pCurrentDepth += 1, pCurrEnvDistance++, pCurrTableDistance++, pCurrImageData++){
var distFromKinect = *pCurrentDepth;
// Remember values
if (saveEnvDistances){
*pCurrEnvDistance = distFromKinect;
}
int distFromTable = (*pCurrEnvDistance – distFromKinect);
*pCurrTableDistance = distFromTable;
if (distFromTable > minimalFingerHeight) {
// We will make pixel gray if it is considered a hand pixel
*pCurrImageData = 128;
}
}}
}
}
}
}

How to improve the solution

Possible there is a way how to copy whole block of memory from the Kinect depth map to EmguCV gray Image. This can be even faster than iterating through all rows and columns, however I was not able to find a way. If you have some better code sample please share in the comments and I will be happy to publish in this article.