Updated 7/3/11 -Edge detection+Antialiasing on RenderTarget!

Post those lines of code you feel like sharing or find what you require for your project here; or simply use them as tutorials.

Updated 7/3/11 -Edge detection+Antialiasing on RenderTarget!

Postby stefbuet » Tue Jul 27, 2010 7:06 pm

[Updated 7/03/2011]
-new edge detection filter also based on normal & depth but now constant edge size even if you zoom out, more exact detection thanks to a 9x9 kernel instead of old 2*2 (cross) and noise removal with sin function.
-----


Here is a shader I've done which is using depth and normals to find edges.
Depth can't be used alone because edges which are inside of an object have a continuous depth, that's why I'm also using normals.

Please, if you find a faster/better may to do it, post it :)
Note : on the screenshot, the floor have strange white stuff, it's because it contain grass ;)

Result :

Image

VS:
Code: Select all
// simple vertex shader
// Author:Stefbuet
// Date:20/04/2010

//This shader can be used by a wide range of pixel shader.
//It is just giving to the PS the texture coords, vertex pos & color.

varying vec4 textureCoord;

void main() {
   
    //texture coordinate'll be interpolated to the PS
    textureCoord=gl_MultiTexCoord0;

    //don't change any vertex position & color
    gl_FrontColor=gl_Color;
    gl_Position=ftransform();
   

}



PS:
Code: Select all

// edge detection pixel shader
// Author:Stefbuet
// Date:14/06/2010
// Last update:07/03/2011

//this shader is doing an edge detection with depth and normal map


uniform sampler2D depthMap;
uniform float SCREENX, SCREENY;

varying vec4 textureCoord;

vec2 offsetArray[8];

void main() {

   vec2 textCoord=textureCoord.xy;

   textCoord.x*=-1.0;
   
   float currentDepth=texture2D(depthMap, -textCoord).w;
   vec3 normal=texture2D(depthMap, -textCoord).xyz*2.0-vec3(1.0,1.0,1.0);

   offsetArray[0] = vec2(1.0/SCREENX, 1.0/SCREENY);
   offsetArray[1] = vec2(1.0/SCREENX, 0.0);
   offsetArray[2] = vec2(1.0/SCREENX, -1.0/SCREENY);
   offsetArray[3] = vec2(0.0, 1.0/SCREENY);
   offsetArray[4] = vec2(0.0, -1.0/SCREENY);
   offsetArray[5] = vec2(-1.0/SCREENX, 1.0/SCREENY);
   offsetArray[6] = vec2(-1.0/SCREENX, 0.0);
   offsetArray[7] = vec2(-1.0/SCREENX, -1.0/SCREENY);

     
   //depth
   float depthDif=0.0;
   float minD=0.005;

   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[0]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[1]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[2]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[3]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[4]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[5]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[6]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[7]).w-currentDepth));

   depthDif/=(1.0+5.0*currentDepth);
   if(depthDif<0.01) depthDif=0;
   depthDif*=25.0;
   depthDif=min(1.0, depthDif);
   depthDif=sin(depthDif*2.0);

   //normals
   float t=1-abs(dot(normal, texture2D(depthMap, -textureCoord.xy+offsetArray[0]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[1]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[2]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[3]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[4]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[5]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[6]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[7]).xyz*2.0-vec3(1.0,1.0,1.0)));
   
   t/=15.0;
   t/=(1.0+2.5*currentDepth);
   if(t<0.16) t=0.0;
   t=sin(t*3.74);

   depthDif+=t;

       
   if(depthDif>0.2) {
       
       gl_FragColor=vec4(1.0,1.0,1.0,1.0);

   }
   else { 
     
       gl_FragColor=vec4(0.0,0.0,0.0,0.0);
     
   }


}



You have to give to the pixel shader the following data :
depthMap : Rendertarget : XYZ = normal, W = depth (view space)
SCREENX, SCREENY : depthMap size, must be floats.

This is a postprocess effect, so you might apply this shader to the quad you're using to do this post processing and put the depthMap texture on it.

Now, here is the antialiasing stuff :
We blur detected edge with a simple 3x3 blur kernel.

Result of the effect :

Image

new pixel shader :

PS :
Code: Select all
// antialiasing pixel shader
// Author:Stefbuet
// Date:14/06/2010
// Last update:07/03/2011

//this shader is doing an edge detection with depth and normal map
//then it blur edges with a more or less good quelity filter (user selected)

uniform sampler2D depthMap;
uniform sampler2D renderMap;
uniform float SCREENX, SCREENY;

varying vec4 textureCoord;

vec2 offsetArray[8];

void main() {

   vec2 textCoord=textureCoord.xy;

   textCoord.x*=-1.0;
   
   float currentDepth=texture2D(depthMap, -textCoord).w;
   vec3 normal=texture2D(depthMap, -textCoord).xyz*2.0-vec3(1.0,1.0,1.0);

   offsetArray[0] = vec2(1.0/SCREENX, 1.0/SCREENY);
   offsetArray[1] = vec2(1.0/SCREENX, 0.0);
   offsetArray[2] = vec2(1.0/SCREENX, -1.0/SCREENY);
   offsetArray[3] = vec2(0.0, 1.0/SCREENY);
   offsetArray[4] = vec2(0.0, -1.0/SCREENY);
   offsetArray[5] = vec2(-1.0/SCREENX, 1.0/SCREENY);
   offsetArray[6] = vec2(-1.0/SCREENX, 0.0);
   offsetArray[7] = vec2(-1.0/SCREENX, -1.0/SCREENY);

     
   //depth
   float depthDif=0.0;
   float minD=0.005;

   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[0]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[1]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[2]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[3]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[4]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[5]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[6]).w-currentDepth));
   depthDif+=min(minD,abs(texture2D(depthMap, -textCoord+offsetArray[7]).w-currentDepth));

   depthDif/=(1.0+5.0*currentDepth);
   if(depthDif<0.01) depthDif=0;
   depthDif*=25.0;
   depthDif=min(1.0, depthDif);
   depthDif=sin(depthDif*2.0);

   //normals
   float t=1-abs(dot(normal, texture2D(depthMap, -textureCoord.xy+offsetArray[0]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[1]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[2]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[3]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[4]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[5]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[6]).xyz*2.0-vec3(1.0,1.0,1.0)));
   t+=1-abs(dot(normal, texture2D(depthMap, -textCoord+offsetArray[7]).xyz*2.0-vec3(1.0,1.0,1.0)));
   
   t/=15.0;
   t/=(1.0+2.5*currentDepth);
   if(t<0.16) t=0.0;
   t=sin(t*3.74);

   depthDif+=t;

       
   if(depthDif>0.2) {
       
       vec4 finalColor=texture2D(renderMap, -textCoord.xy);
       for(int i=0; i<8; i++)
           finalColor+=texture2D(renderMap, -textCoord.xy+offsetArray[i]);
       
       finalColor/=8.0;
       gl_FragColor=finalColor;

   }
   else { 
     
       gl_FragColor=texture2D(renderMap, -textCoord);
     
   }


}



You have only to give a new data to the shader:
renderMap : render target which contain your scene.

This method is used by Stalker to do antialiasing.
See http://http.developer.nvidia.com/GPUGem ... ter09.html after the deferred lighting part.

I saw that bluring far away things may let them too blurred. Maybe modulate the blur in function of the depth of pixels is a good idea, but I haven't tested it yet!

Hope you will enjoy it :P
Last edited by stefbuet on Mon Mar 07, 2011 7:03 am, edited 2 times in total.
User avatar
stefbuet
Competition winner
 
Posts: 495
Joined: Sun Dec 09, 2007 4:13 pm
Location: france

Postby slavik262 » Tue Jul 27, 2010 9:11 pm

Cool. I was thinking of writing a similar effect - I'll definitely use this as a basis for mine.
Last edited by slavik262 on Tue Jul 27, 2010 11:14 pm, edited 1 time in total.
User avatar
slavik262
 
Posts: 753
Joined: Sun Nov 22, 2009 9:25 pm
Location: Wisconsin, USA

Postby andres » Tue Jul 27, 2010 9:14 pm

Very good. Thanks!
Prof. Andres Jessé Porfirio
Federal Technological University of Parana (UTFPR)
www.andresjesse.com
http://irrrpgbuilder.sourceforge.net

Image
User avatar
andres
Competition winner
 
Posts: 78
Joined: Tue Jul 08, 2008 5:18 pm
Location: Guarapuava/Brazil

Postby wing64 » Wed Jul 28, 2010 8:42 am

Good job. :D
wing64
Competition winner
 
Posts: 240
Joined: Wed Jul 23, 2008 2:35 am
Location: Thailand

Postby greenya » Wed Jul 28, 2010 10:12 am

Second screenshot pair:
anti-aliasing stuff:
48 fps with OFF and 230 fps with ON --- incredible indeed :roll:
greenya
 
Posts: 1007
Joined: Sun Jan 21, 2007 1:46 pm
Location: Ukraine

Postby slavik262 » Wed Jul 28, 2010 11:59 am

greenya wrote:Second screenshot pair:
anti-aliasing stuff:
48 fps with OFF and 230 fps with ON


:shock: How is that possible?
User avatar
slavik262
 
Posts: 753
Joined: Sun Nov 22, 2009 9:25 pm
Location: Wisconsin, USA

Postby ACE247 » Wed Jul 28, 2010 1:18 pm

Whats going on here? 48 without and 230 with??? Some hardware fps boost.
And i see you didn't mix them up, since the blurred vs unblurred is clearly visible.
ACE247
 
Posts: 704
Joined: Tue Mar 16, 2010 12:31 am

Postby DtD » Wed Jul 28, 2010 4:18 pm

lol on the FPS. Looks great!
User avatar
DtD
 
Posts: 264
Joined: Mon Aug 11, 2008 7:05 am
Location: Kansas

Postby ACE247 » Thu Jul 29, 2010 3:40 pm

Hmm.. I just got the idea that this might be usefull for doing something kinda like soft particles for when they intersect with other geometry like the terrain to blur the intersect edges out. Or am I wrong?
ACE247
 
Posts: 704
Joined: Tue Mar 16, 2010 12:31 am

Postby booe » Thu Jul 29, 2010 4:58 pm

Hum, stefbuet, you must have unwittingly switched the "on"/"off" labels.
The "on" label should belong to screenie with 48 fps, and "off" label to one with 230.
booe
 
Posts: 76
Joined: Thu Jul 29, 2010 2:12 pm

Postby ACE247 » Thu Jul 29, 2010 5:43 pm

No he hasn't, you can clearly see the smoothed out edges in the different screenshots.
ACE247
 
Posts: 704
Joined: Tue Mar 16, 2010 12:31 am

Postby slavik262 » Thu Jul 29, 2010 6:16 pm

ACE247 wrote:Hmm.. I just got the idea that this might be usefull for doing something kinda like soft particles for when they intersect with other geometry like the terrain to blur the intersect edges out. Or am I wrong?


It's funny you should say that; I was looking at some soft particle code the other day from the DirectX SDK. Soft particles work by rendering the particles to a seperate RTT. You also render the depth of the particles, and then when you add the particles back into the final frame, you compare the depth of the frame against the depth of the particles. You only draw the particles if they are in front of the rest of the frame, so that way they don't clip into geometry.
User avatar
slavik262
 
Posts: 753
Joined: Sun Nov 22, 2009 9:25 pm
Location: Wisconsin, USA

Postby stefbuet » Thu Jul 29, 2010 6:24 pm

Ok, here is the FPS answer :
3dsmax was opening when I took the screenshot. :P
The shader don't change the FPS, however as I'm on a geforce 260GTX, it might change a little for lower spec graphic card.
User avatar
stefbuet
Competition winner
 
Posts: 495
Joined: Sun Dec 09, 2007 4:13 pm
Location: france

Postby slavik262 » Thu Jul 29, 2010 7:26 pm

Since it's completely dependant on the pixel shader pipeline, performance is mostly a hardware question. On a decent graphics card it won't slow anything down, but it will choke on crap hardware (which is why games usually make post-processing optional)
User avatar
slavik262
 
Posts: 753
Joined: Sun Nov 22, 2009 9:25 pm
Location: Wisconsin, USA

Postby devsh » Tue Nov 30, 2010 2:39 pm

you should be careful with normal map edge detection cause otherwise you will end up blurring bumpmapped surfaces so be sure to attenuate your normal found edges by the depth edge detection.. I could think of a very complicated algo but it would be slightly inefficient... I made my own that also takes the color buffer into account so I might share it in some time
We chose to stream mesh data from Multiple OpenGL Contexts in many threads and do the other things, not because they are easy, but because they are hard! - JFK
User avatar
devsh
Competition winner
 
Posts: 1766
Joined: Tue Dec 09, 2008 6:00 pm
Location: UK

Next

Return to Code Snippets

Who is online

Users browsing this forum: zerosang and 0 guests