I’m working on a project that uses the amazing and wonderful as3nui.com to make something delightful with Away3d and Kinect. I’d like to show the Kinect video stream underneath the Away3d render.

I understand that Away3d 4.0 writes to the stage3d layer of Flash, and the only thing below it is stageVideo. It doesn’t seem like I can write arbitrary bitmapData to stageVideo (and the Kinect doesn’t show up as a camera), so it seems like I have to write the background image through Away3d (or not? Please tell me if there’s a better way).

I’ve tried to set a bitmap texture as view3d.background, but it scales the image. My bitmap is 640x480 (streamed from Kinect) and the bitmap image needs to be a power of 2, so I’ve set it to 1024x512. view3d.background dutifully scaled the image to fill the space—so my 4:3 video is distorted by the GPU defined ratio (i.e. 512x512, 1:1 or 1024x512, 2:1). This doesn’t work, since the image needs to be calibrated to the Kinect dimensions. Am I missing something? I’m sure I am, but I’m too addled to figure it out.

I’ve also tried to make a plane in the background of my space, and apply the bitmap image to it, but I can’t figure out how to calibrate it properly. I’m sure this is solvable, and I plea to your greater wisdom!

Thank you!

I’ve solved my problem and will describe it hoping that it helps other lonely coders:

I have a 640x480 video stream fed to me as BitmapData at 30fps. I want to set the BitmapData as the background of my Away3d view, so the user sees the 3d interaction overlaid the video of him/her. My Away3d view is also 640x480.

You can set a BitmapTexture to be the background of a View3D, but the bitmap needs to be a power of 2 (i.e. 512x512). Because 640x480 !=512x512, I had a problem. I tried writing the 640x480 as-is to the BitmapTexture, but it appeared distorted.

What I didn’t realize—and may be obvious to 3d regulars—is that a material is stretched to fill the entire surface. Thus my 512x512 bitmap was stretched to fit my 640x480 viewport. To fix it, you have to squeeze the 640x480 image into 512x512, so it looks right when it’s stretched into its final position.

In other words:

//Set up view
viewport = new View3D(); 
viewport.width = 640
viewport.height = 480

//Set up a container for the image data
rgb = new BitmapData(512, 512,false,0x00FF0000);

//Set up matrix to scale the image data
rgbMatrix = new Matrix();
rgbMatrix.scale(512/640,512/480);

//Set texture to viewport background
rgbTexture = new BitmapTexture(rgb);
viewport.background = rgbTexture;

When you get your new RGB values:

private function onRGB(event:CameraFrameEvent):void { 
  //Draw new rgb bitmapData into our container, applying the matrix
  //so it will look right when applied to the background
  rgb.draw(event.data.frame,rgbMatrix,null,null,null,false);
}

And on your tick:

private function onFrame(event:Event=null):void {
  rgbTexture.bitmapData = rgb //Update bitmapdata
  rgbTexture.invalidateContent()  //Force update on texture
  viewport.render();
}

Hope this helps someone! If it’s horribly wrong or there’s a better way, please let me know. With simple models, this is running the as3nui skeleton recognition, video image, depth image @ 30fps.

doolittle..

thanx a lot man, I was facing the same prob for past few weeks and you made it really easy… thanx