[TUTORIAL] Depth maps and Ultrabooks

    Went to a really great hack-a-thon this past weekend at the Sacramento Hacker Lab to help coach some folks through working with the Perceptual Computing SDK and got to see some really cool work being done, everything from a next-generation theremin to a telepresence bot, all powered by the Creative 3D Camera and Perceptual Computing SDK.  Does me good to actually get out into the community and see people just dive right in and start building stuff.  Compound that with the GDC Dev Day that personally I think went amazingly well (standing room only at one point!) and it's been a good GDC for Perceptual Computing so far.  But now comes the really hard part, which is that PerC needs to not become a victim of its own success.  As the technology gets into more hands, now it becomes about not burning through goodwill by breaking features, being uncommunicative, or not keeping up with the ecosystem.  But I digress...

    Wanted to share a little Unity tip I got asked about a few times during the hack-a-thon, and that's how to visualize the depth map.  The SDK ships with a sample for visualizing the label map, and visualizing the color map is a fairly trivial change, but visualizing the depth map requires a little bit of doing.  It's actually pretty trivial from a working standpoint, so let's take a look at what's required.

    To get a depth map into a usable Texture2D, the basic flow is:

• Grab the depth buffer into a short array
• Walk the array of depth values and and remap them into 0-1 range
• Store the remapped value in a Color array
• Load the Color array into a Texture2D

    If that seems really simple, fear not, it actually is, so let's take a look at some code and see how we accomplish this.  Here's a really simple Unity behavior that populates the texture object from the depth map.  I'll leave assigning the texture as an exercise to the readers:

using UnityEngine;
using System.Collections;

public class Test : MonoBehaviour
{
    private PXCUPipeline mSession;
    private int[] mDepthSize;
    private short[] mDepthBuffer;
    private int mSize;

    private Texture2D mDepthMap;
    private Color[] mDepthPixels;

    void Start()
    {
        mDepthSize = new int[2];
        mSession = new PXCUPipeline();
        mSession.Init(PXCUPipeline.Mode.DEPTH_QVGA);
        mSession.QueryDepthMapSize(mDepthSize);
        mSize = mDepthSize[0]*mDepthSize[1];

        mDepthMap = new Texture2D(mDepthSize[0], mDepthSize[1], TextureFormat.ARGB32, false);
        mDepthBuffer = new short[mSize];
        mDepthPixels = new Color[mSize];
        for(int i=0;i<mSize;++i)
        {
            mDepthPixels[i] = Color.black;
        }
    }

    void Update()
    {
        if(mSession.AcquireFrame(false))
        {
            mSession.QueryDepthMap(mDepthBuffer);
            for(int i=0;i<mSize;++i)
            {
                float v = 1.0f-lmap((float)mDepthBuffer[i],0,1800.f,0,1.f);
                mDepthPixels[i] = new Color(v,v,v);
                mDepthMap.SetPixels(mDepthPixels);
                mDepthMap.Apply();
            }
            mSession.ReleaseFrame();
        }
    }

    float lmap(float val, float min0, float max0, float min1, float max1)
    {
        return min1 + (val-min0)*(max1-min1)/(max0-min0);
    }
}

    So like i said, fairly simple, albeit verbose technique, but should be fairly easy to wrap it up into a simple function for quick future use.  This same technique can also be used to visualize the IR map with some very minor tweaks.  I've actually been doing alot of stupid depth map tricks the last few days.  I'm at GDC all this week so I'm not sure how much dev time I'll get to be able to polish a few more of these up but maybe the weekend'll afford me some cycles if i'm not in full on crash out recovery mode...