Kinect v1 SDK C++ - 2. Kinect Depth Data

Have a Kinect v2? Head over to the Kinect v2 Tutorial 2. This tutorial is for the v1 Kinect SDK.

Goals: Learn how to get depth data from a kinect and what the format of the data is.
Source: View Source      Download: 2_Depth.zip

Overview

This is a very short tutorial - there are only two real changes to the RGB code.

I have also refactored the SDL and Glut components into separate files to focus more on the Kinect specifics.

Contents

  1. Kinect Code
  2. Display Frameworks

Kinect Code

Kinect Initialization

To get the depth data from the kinect, simply change the arguments to NuiImageStreamOpen(). The first argument is now NUI_IMAGE_TYPE_DEPTH, telling the Kinect that we now want depth images instead of RGB images. (For clarity we also changed the name of the Handle to reflect this)

When you're using the Kinect for Windows sensor (as opposed to the XBox Kinect) you can also enable Near Mode. Near Mode allows the Kinect to be more sensitive to closer objects (say from 50cm to 200cm), whereas without near mode it is sensitive from 80cm to 400cm.

To enable near mode, pass in the flag NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE as the third argument. 

// NEW VARIABLE
HANDLE depthStream;

bool initKinect() {
    // Get a working kinect sensor
    int numSensors;
    if (NuiGetSensorCount(&numSensors) < 0 || numSensors < 1) return false;
    if (NuiCreateSensorByIndex(0, &sensor) < 0) return false;

    // Initialize sensor
    sensor->NuiInitialize(NUI_INITIALIZE_FLAG_USES_DEPTH | NUI_INITIALIZE_FLAG_USES_COLOR);

        // --------------- START CHANGED CODE -----------------
    sensor->NuiImageStreamOpen(
        NUI_IMAGE_TYPE_DEPTH,                     // Depth camera or rgb camera?
        NUI_IMAGE_RESOLUTION_640x480,             // Image resolution
        NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE,   // Image stream flags, e.g. near mode
        2,      // Number of frames to buffer
        NULL,   // Event handle
        &depthStream);
        // --------------- END CHANGED CODE -----------------
    return sensor;
}
For more information about near mode, see this official blog post.

Getting a depth frame from the Kinect

We'll display the depth image from the kinect in grayscale. Each pixel will just be the pixel's distance from the kinect (in millimeters) mod 256.

The only thing to note here is the NuiDepthPixelToDepth function - each pixel in the actual depth map has both depth information and player information (i.e. which player can be associated with that pixel). Calling this function returns the depth in millimeters at that pixel.

The depth data is 16 bits, so we use a USHORT to read it in. 

        const USHORT* curr = (const USHORT*) LockedRect.pBits;
        const USHORT* dataEnd = curr + (width*height);

        while (curr < dataEnd) {
            // Get depth in millimeters
            USHORT depth = NuiDepthPixelToDepth(*curr++);

            // Draw a grayscale image of the depth:
            // B,G,R are all set to depth%256, alpha set to 1.
            for (int i = 0; i < 3; ++i)
                *dest++ = (BYTE) depth%256;
            *dest++ = 0xff;
        }

That's all the Kinect code! The rest is just how to get it onscreen.

Display Frameworks

I refactored the code so that the glut and SDL specifics are bundled up into an init() function, a draw() function, and an execute() function.

To use one or the other, simply change the include at the top of main.cpp to use either sdl.h or glut.h. Make sure you have the appropriate includes and links in the Project Properties!

The End! Build and run, making sure that your Kinect is plugged in. You should see a window containing a video stream of what your Kinect sees, in grayscale.
Previous: Basics

Next: Kinect Point Cloud