Away3D Forum Depth map in custom shader

kurono, Sr. Member
Posted: 03 August 2012 12:37 AM Total Posts: 103

Hi there!

I want to sample depth map for further use as the filter3d for view3D.
Here’s the code of filter’s task:

override public function activate(stage3DProxy : Stage3DProxy, camera3D : Camera3D, depthTexture : Texture) : void
  {
   stage3DProxy.setTextureAt(1, depthTexture); // work with depth
   stage3DProxy.context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 0, _rgbData, 2);
  }
  
  override public function deactivate(stage3DProxy : Stage3DProxy) : void
  {
   stage3DProxy.setTextureAt(1, null);
  } 

And then in override protected function getFragmentCode():

return "mov ft2, v0       \n" + // ft2 = pos
     "tex ft1, v0, fs1 <2d,nearest>   \n" + // get depth map
     "tex ft0, v0, fs0 <2d,nearest>   \n" + // get texture map
     "mov ft3, ft0       \n" + // just init new variable
     "mov oc, ft3       \n"; 

But “tex ft1, v0, fs1 <2d,nearest>” always causes black screen, but as you can see I’ve just put texture of view3d to output color.

Please help!

Richard Olsson, Administrator
Posted: 03 August 2012 08:28 AM Total Posts: 1192 [ # 1 ]

What do you mean with “always causes a black screen”? Is it when you add that line into your shader that you get a black screen, even though you do not use ft1 later on in the shader? Or do you mean that the fragment stored in ft1 is always black?

Are you sure that there is a depth texture passed into your activate() function? Are you passing true to the task parent class constructor argument (requiresDepthRender)?

kurono, Sr. Member
Posted: 03 August 2012 09:07 AM Total Posts: 103 [ # 2 ]

“Is it when you add that line into your shader that you get a black screen” - that’s right.

Yes, I forgot to add “super(true)” in my filter’s task! So, depth map was always null (i.e. requiresDepthRender = false). Thanks a lot, Richard!!!

—-

One more question!
I’ve slightly modified a Filter3DDepthOfFFieldTask because want to encode depth from depth map.

Unchanged:

_data = Vector.<Number>([ 0, 0, 0,  _focusDistance,  // fc0[xyzw]
          0, 0, 0,   0,     // fc1[xyzw]
          _range, 0, 0, 0,     // fc2[xyzw]
          1.0, 1 / 255.0, 1 / 65025.0, 1 / 16581375.0]); // fc3[xyzw] -> ARGB 

// sample depth, unpack & get blur amount (offset point + step size)
   code =  "tex ft0, v0, fs1 <2d, nearest> \n" +
     "dp4 ft1.z, ft0, fc3    \n" +
     "sub ft1.z, ft1.z, fc1.z   \n" + // d = d - f
     "div ft1.z, fc1.w, ft1.z   \n" + // screenZ = -n*f/(d-f)
     "sub ft1.z, ft1.z, fc0.w   \n" + // screenZ - dist
     "div ft1.z, ft1.z, fc2.x   \n" + // (screenZ - dist)/range

     "abs ft1.z, ft1.z     \n" + // abs(screenZ - dist)/range
     "sat ft1.z, ft1.z     \n" + // sat(abs(screenZ - dist)/range)
     "mul ft6.xy, ft1.z, fc0.xy   \n" +
     "mul ft7.xy, ft1.z, fc1.xy   \n"; 

Then I commented blur code and simply multiply view3d’s texture by depth as follows:

code += "tex ft2, v0, fs0 <2d,linear,clamp> \n"; // sample view: rgb
   code += "sub ft1.z, fc3.x, ft1.z   \n"; // depth = 1 - depth
   code += "mul ft2.xyzw, ft1.z, ft2.xyzw  \n"; // rgb = rgb * depth
   code += "mov oc, ft2      \n"; // return 

It seems that depthmap’s and view3d’s sizes are slightly different. That’s why texture is shifted from another one. But why they’re differ? Need help! Please!

—-

Hmm, if view3d is square shaped and its dimensions are the power of two, everything’s ok. So, I have to use modified coords for sampling instead of simple v0. (There’re no lights on second picture)

kurono, Sr. Member
Posted: 03 August 2012 02:10 PM Total Posts: 103 [ # 3 ]

I cant’ adjust positions of both textures :(
Depth map was sampled at scaled and shifted position (ft7):

code =  "mov ft7, v0      \n"; // pos
   code += "mul ft7.xy, v0.xy, fc0.xy   \n"; // scale
   code += "sub ft7.xy, ft7.xy, fc1.xy   \n"; // try to shift
   // sample depth, unpack & get blur amount (offset point + step size)
   code += "tex ft0, ft7, fs1 <2d, nearest> \n" + 

Where fc0 and fc1 are respectively:

_data[0] = 1;
   _data[1] = 800 / 600;
   _data[4] = 0;
   _data[5] = 0; 

And then sample view3d as it was for previous post:

code += "tex ft2, v0, fs0 <2d,linear,clamp> \n"; // sample view at pos: rgb
   code += "sub ft1.z, fc3.x, ft1.z    \n"; // depth = 1 - depth
   code += "mul ft2.xyzw, ft1.z, ft2.xyzw   \n"; // rgb = rgb * depth
   code += "mov oc, ft2       \n"; // return 

So, the idea is to scale sampling point according to size of view3d (800x600).

As the result:
1) Scaling seems to be ok… Maybe.
2) Shift is bad, so I temporary set it to zero (_data[4], _data[5]).

kurono, Sr. Member
Posted: 04 August 2012 04:05 PM Total Posts: 103 [ # 4 ]

Ok, I founded the right way to scale depth map according to view3d:

protected function updateConstants():void
  {
   _data = Vector.<Number>([
    1.0,            1.0,           1.0,         0,    // fc0[xyzw]
    1024/800,   1024/600,     1.0,        1.0,    // fc1[xyzw]
    1.0,    1/255.0,     1/65025.0,   1/16581375.0 // fc2[xyzw], ARGB
   ]);
  } 

Then:

return "mov ft2, v0       \n" + // ft2 = pos
     /*"add ft2.x, ft2.x, fc1.z    \n" + // shift x
     "add ft2.y, ft2.y, fc1.w    \n" + // shift y*/
     "mul ft2.xy, ft2.xy, fc1.xy    \n" + // scale
     "tex ft1, ft2, fs1 <2d,nearest>   \n" + // get depth map
     "tex ft0, v0, fs0 <2d,nearest>   \n" + // get texture map
     "mul ft1.xyz, ft0.xyz, ft1.xyz   \n" +
     "mov oc, ft1       \n"; 

The result is attached: depth map’s and view3d’s sizes are equal, but they’re shifted.

But the strange thing is that I can’t shift depth texture properly (that’s why this part of code is commented). Even if shift equals 1.0 (_data[7], _data[8]), I can’t see my depth texture (see 2nd screenshot)! So, the question is: how to arbitrary shift depth texture using AGAL?

P.S.
In PixelBender everytnig works fine, but i want it in pure AGAL as the filter3d.

<languageVersion : 1.0;>

kernel Resizer
<   namespace : "";
    vendor : "";
    version : 1;
>
{
    parameter float3 shift
    <
        minValue:       float3(-512.0,-512.0,-180.0);
        maxValue:       float3(512.0,512.0,180.0);
        defaultValue:   float3(0,0,1);
    >;
    
    parameter float2 scale
    <
        minValue:       float2(64,64);
        maxValue:       float2(1024,1024);
        defaultValue:   float2(800,600);
    >;
    
    input image4 src;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        float2 point = outCoord() - float2(shift.x,shift.y);
        point = float2(point.x * 1024.0/scale.x, point.y * 1024.0/scale.y);
        dst = sampleNearest(src, point);
    }
} 

kurono, Sr. Member
Posted: 08 August 2012 03:48 PM Total Posts: 103 [ # 5 ]

Bump…

Depth map in custom shader Software: Away3D 4.x

kurono, Sr. Member Posted: 03 August 2012 12:37 AM Total Posts: 103 Hi there! I want to sample depth map for further use as the filter3d for view3D. Here’s the code of filter’s task: `override public function activate(stage3DProxy : Stage3DProxy, camera3D : Camera3D, depthTexture : Texture) : void { stage3DProxy.setTextureAt(1, depthTexture); // work with depth stage3DProxy.context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 0, _rgbData, 2); } override public function deactivate(stage3DProxy : Stage3DProxy) : void { stage3DProxy.setTextureAt(1, null); }` And then in override protected function getFragmentCode(): `return "mov ft2, v0 \n" + // ft2 = pos "tex ft1, v0, fs1 <2d,nearest> \n" + // get depth map "tex ft0, v0, fs0 <2d,nearest> \n" + // get texture map "mov ft3, ft0 \n" + // just init new variable "mov oc, ft3 \n";` But “tex ft1, v0, fs1 <2d,nearest>” always causes black screen, but as you can see I’ve just put texture of view3d to output color. Please help!
Richard Olsson, Administrator Posted: 03 August 2012 08:28 AM Total Posts: 1192 [ # 1 ] What do you mean with “always causes a black screen”? Is it when you add that line into your shader that you get a black screen, even though you do not use ft1 later on in the shader? Or do you mean that the fragment stored in ft1 is always black? Are you sure that there is a depth texture passed into your activate() function? Are you passing true to the task parent class constructor argument (requiresDepthRender)?
kurono, Sr. Member Posted: 03 August 2012 09:07 AM Total Posts: 103 [ # 2 ] “Is it when you add that line into your shader that you get a black screen” - that’s right. Yes, I forgot to add “super(true)” in my filter’s task! So, depth map was always null (i.e. requiresDepthRender = false). Thanks a lot, Richard!!! —- One more question! I’ve slightly modified a Filter3DDepthOfFFieldTask because want to encode depth from depth map. Unchanged: `_data = Vector.<Number>([ 0, 0, 0, _focusDistance, // fc0[xyzw] 0, 0, 0, 0, // fc1[xyzw] _range, 0, 0, 0, // fc2[xyzw] 1.0, 1 / 255.0, 1 / 65025.0, 1 / 16581375.0]); // fc3[xyzw] -> ARGB` // sample depth, unpack & get blur amount (offset point + step size) code = "tex ft0, v0, fs1 <2d, nearest> \n" + "dp4 ft1.z, ft0, fc3 \n" + "sub ft1.z, ft1.z, fc1.z \n" + // d = d - f "div ft1.z, fc1.w, ft1.z \n" + // screenZ = -nf/(d-f) "sub ft1.z, ft1.z, fc0.w \n" + // screenZ - dist "div ft1.z, ft1.z, fc2.x \n" + // (screenZ - dist)/range "abs ft1.z, ft1.z \n" + // abs(screenZ - dist)/range "sat ft1.z, ft1.z \n" + // sat(abs(screenZ - dist)/range) "mul ft6.xy, ft1.z, fc0.xy \n" + "mul ft7.xy, ft1.z, fc1.xy \n"; Then I commented blur code and simply multiply view3d’s texture by depth as follows: `code += "tex ft2, v0, fs0 <2d,linear,clamp> \n"; // sample view: rgb code += "sub ft1.z, fc3.x, ft1.z \n"; // depth = 1 - depth code += "mul ft2.xyzw, ft1.z, ft2.xyzw \n"; // rgb = rgb depth code += "mov oc, ft2 \n"; // return` It seems that depthmap’s and view3d’s sizes are slightly different. That’s why texture is shifted from another one. But why they’re differ? Need help! Please! —- Hmm, if view3d is square shaped and its dimensions are the power of two, everything’s ok. So, I have to use modified coords for sampling instead of simple v0. (There’re no lights on second picture)
kurono, Sr. Member Posted: 03 August 2012 02:10 PM Total Posts: 103 [ # 3 ] I cant’ adjust positions of both textures :( Depth map was sampled at scaled and shifted position (ft7): `code = "mov ft7, v0 \n"; // pos code += "mul ft7.xy, v0.xy, fc0.xy \n"; // scale code += "sub ft7.xy, ft7.xy, fc1.xy \n"; // try to shift // sample depth, unpack & get blur amount (offset point + step size) code += "tex ft0, ft7, fs1 <2d, nearest> \n" +` Where fc0 and fc1 are respectively: `_data[0] = 1; _data[1] = 800 / 600; _data[4] = 0; _data[5] = 0;` And then sample view3d as it was for previous post: `code += "tex ft2, v0, fs0 <2d,linear,clamp> \n"; // sample view at pos: rgb code += "sub ft1.z, fc3.x, ft1.z \n"; // depth = 1 - depth code += "mul ft2.xyzw, ft1.z, ft2.xyzw \n"; // rgb = rgb * depth code += "mov oc, ft2 \n"; // return` So, the idea is to scale sampling point according to size of view3d (800x600). As the result: 1) Scaling seems to be ok… Maybe. 2) Shift is bad, so I temporary set it to zero (_data[4], _data[5]).
kurono, Sr. Member Posted: 04 August 2012 04:05 PM Total Posts: 103 [ # 4 ] Ok, I founded the right way to scale depth map according to view3d: `protected function updateConstants():void { _data = Vector.<Number>([ 1.0, 1.0, 1.0, 0, // fc0[xyzw] 1024/800, 1024/600, 1.0, 1.0, // fc1[xyzw] 1.0, 1/255.0, 1/65025.0, 1/16581375.0 // fc2[xyzw], ARGB ]); }` Then: return "mov ft2, v0 \n" + // ft2 = pos /"add ft2.x, ft2.x, fc1.z \n" + // shift x "add ft2.y, ft2.y, fc1.w \n" + // shift y/ "mul ft2.xy, ft2.xy, fc1.xy \n" + // scale "tex ft1, ft2, fs1 <2d,nearest> \n" + // get depth map "tex ft0, v0, fs0 <2d,nearest> \n" + // get texture map "mul ft1.xyz, ft0.xyz, ft1.xyz \n" + "mov oc, ft1 \n"; The result is attached: depth map’s and view3d’s sizes are equal, but they’re shifted. But the strange thing is that I can’t shift depth texture properly (that’s why this part of code is commented). Even if shift equals 1.0 (_data[7], _data[8]), I can’t see my depth texture (see 2nd screenshot)! So, the question is: how to arbitrary shift depth texture using AGAL? P.S. In PixelBender everytnig works fine, but i want it in pure AGAL as the filter3d. <languageVersion : 1.0;> kernel Resizer < namespace : ""; vendor : ""; version : 1; > { parameter float3 shift < minValue: float3(-512.0,-512.0,-180.0); maxValue: float3(512.0,512.0,180.0); defaultValue: float3(0,0,1); >; parameter float2 scale < minValue: float2(64,64); maxValue: float2(1024,1024); defaultValue: float2(800,600); >; input image4 src; output pixel4 dst; void evaluatePixel() { float2 point = outCoord() - float2(shift.x,shift.y); point = float2(point.x * 1024.0/scale.x, point.y * 1024.0/scale.y); dst = sampleNearest(src, point); } }
kurono, Sr. Member Posted: 08 August 2012 03:48 PM Total Posts: 103 [ # 5 ] Bump…

Depth map in custom shader

Login

Away3D Forum

Member Login