Problem with GLSL shader pixel lighting...

[christianclavet]

Hi,

First, I want to warn everybody that I'm really noob with shaders, this is my first attemp to update a shader by reading tutorials about GLSL. Also I don't understand math equations, but still can understand when I'm explained in code.

I was able to work on the terrain shader made by Andres that blend 4 textures and use the height value to set the blend factor (that part was made by Andres and it work really well). I just added more parameters to it and it still worked fine.

Since 2 day, I'm trying to incorporate lighting (pixel based lighting) and was successful yesterday, by having the terrain support 1 directionnal light (fake sun using the ambiant light color) and a point light (at the camera position to light the scene when it's dark. (night scenes)

Everything work fine, except that my terrain is made by object "tiles", each tile is a mesh that has the shader applied on it. And the problem is only there: The lighting seem to affect all the "land tiles" the same, I see lighting problem at the edges.

Here is a screenshot:

If you see from the screenshot, if you look at the mountain in front, the lighting seem to start to fade to dark, then start bright from the edge of the other mesh. Is there anything I could do in the shader to stop the edge problem?

Here is the current vertex shader:

Code: Select all

varying vec3 normal;
varying vec3 position;

void main(void)
{
   //gl_Position = ftransform();
   gl_Position =  gl_ModelViewProjectionMatrix * gl_Vertex;
   gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
   gl_FogFragCoord = gl_Position.z;
   normal = gl_Normal.xyz;
   position = gl_Vertex.xyz;
}

Here is the pixel (fragment) shader:

Code: Select all

uniform sampler2D terrainLayer0;
uniform sampler2D terrainLayer1;
uniform sampler2D terrainLayer2;
uniform sampler2D terrainLayer3;
uniform sampler2D terrainLayer4;

uniform float plateau;
uniform int terrainTextureScale;
uniform int terrainScale;

uniform bool editingTerrain;

uniform vec4 AmbientLight;
uniform vec3 mLightPos;
uniform vec3 mCamPos;

varying vec3 normal;
varying vec3 position;

void main()
{
    int scale = terrainScale / 4;
   if (terrainScale==1)
      scale = 1;
      
   vec2 texCoord = vec2(gl_TexCoord[0]);
   
   vec4 tex0    = texture2D( terrainLayer0, texCoord.xy*terrainTextureScale );
   vec4 tex1    = texture2D( terrainLayer1, texCoord.xy*terrainTextureScale );
   vec4 tex2    = texture2D( terrainLayer2, texCoord.xy*terrainTextureScale );
   vec4 tex3    = texture2D( terrainLayer3, texCoord.xy*terrainTextureScale );

   tex1 = mix( tex1, tex0, min(1-normal.y,1) );
   tex2 = mix( tex1, tex2, (position.y/scale));
   
   vec4 tex10;
   
   if(position.y >= 0)
   {
     tex10 = mix( tex1, tex3, (position.y/scale));
     tex10 = mix( tex10, tex2, (min(1-normal.y-0.2,1)) );
   }
   else
   {
     tex10 = mix( tex1, tex2, min(1-normal.y-0.2,1));
     tex10 = mix( tex10, tex0, min(1,-(position.y/scale)*10));
   }
   
   if(position.y>(plateau-2.5) && position.y<(plateau+2.5) && editingTerrain) tex10*=vec4(1,0.6,0.4,1);
      
   // Point light with attenuation calculation   
   vec4 diffuse;
   vec3 norm = normalize(normal);
   vec3 lightVector = mLightPos - position;
   float dist = length(lightVector);
   float attenuation = 1.0 /( gl_LightSource[0].constantAttenuation +
                        gl_LightSource[0].linearAttenuation * dist +
                        gl_LightSource[0].quadraticAttenuation * dist * dist);
                        
   lightVector = normalize(lightVector);
   float nxDir = max(0.0, dot(norm, lightVector));
   
   // Directional light no attenuation
   vec4 sunDiffuse;
   vec3 sunVector = normalize(vec3(0,500,0) - position);
   float sunDir = max(0.0, dot(norm, sunVector));
   
   //Seem to cause seams with lighting
   diffuse = (gl_LightSource[1].diffuse * sunDir) + (gl_LightSource[0].diffuse * nxDir * attenuation);
   
   //diffuse = AmbientLight +(gl_LightSource[0].diffuse * nxDir * attenuation);
      
      

   
   // Flat rendering with FOG possible   
     //vec4 finalColor = tex10;
   
   // Rendering with 1 point light source
   vec4 finalColor = gl_LightSource[0].ambient + (diffuse * vec4(tex10.rgb, 1.0));
   
   float fog = (gl_Fog.end - gl_FogFragCoord) * gl_Fog.scale;
   gl_FragColor = mix(gl_Fog.color,finalColor, fog);
   //mix(gl_Fog.color,finalColor, fog);
}

[Klunk]

that looks like an error in the mesh normals eg they do not match across the tile edge.

[christianclavet]

Thanks! I'll look if is there a way to recalculate the normals once we modify the geometry.

[HerrAlmanack]

If I will understand what you are method. this may be what is occuring



maybe just sweep through the geometry and combine those duplicate vertices? averaging the normals along the way for even smoother lighting

[christianclavet]

Ok, That probably explain what is happening. I would have to recalculate the normals for each edge vertices by averaging the surrounding vertices around.

Humm... Does anybody know how I could do this?

This should be done at the "merging" of the tile, here the code were using now. We can't really merge the geometry, so we're overlapping the edge vertices.

Code: Select all

void TerrainTile::mergeToTile(TerrainTile* tile)
{
    if(!tile)
    {
        #ifdef APP_DEBUG
        cout << "DEBUG : TERRAIN TILE : MERGE FAILED, TILE IS NULL: " << endl;
        #endif
        return;
    }
    else
    {
        IMeshBuffer* meshBuffer = ((IMeshSceneNode*)node)->getMesh()->getMeshBuffer(0);
        IMeshBuffer* neighborBuffer = tile->getMeshBuffer();

        S3DVertex* mb_vertices = (S3DVertex*) meshBuffer->getVertices();
        S3DVertex* n_mb_vertices = (S3DVertex*) neighborBuffer->getVertices();

        u16* mb_indices  = meshBuffer->getIndices();
        u16* n_mb_indices  = neighborBuffer->getIndices();

        for (unsigned int j = 0; j < meshBuffer->getVertexCount(); j += 1)
        {
            for (unsigned int i = 0; i < neighborBuffer->getVertexCount(); i += 1)
            {
                vector3df realPos = mb_vertices[j].Pos*(scale/nodescale) + node->getPosition();
                vector3df nRealPos = n_mb_vertices[i].Pos*(scale/nodescale) + tile->getNode()->getPosition();

                if((realPos.X == nRealPos.X) && (realPos.Z == nRealPos.Z))
                {
                    mb_vertices[j].Pos.Y = n_mb_vertices[i].Pos.Y;
                }
            }
        }

        smgr->getMeshManipulator()->recalculateNormals(((IMeshSceneNode*)node)->getMesh(),true);
      ((IMeshSceneNode*)node)->getMesh()->setDirty();
    }
}

This match the position of the Y for mesh vertex with it surrounding, and recalculate the normals. Since the edge vertices are overlapping, how can I set the normal to average the surrounding values for it's determination?

[HerrAlmanack]

since there are four vertices overlapping at each tile "corner"

try this, I use it for a physics character controller but it should do the same

Code: Select all


irr::core::vector3df* norms;
irr::core::vector3df final_norm(0.0f,0.0f,0.0f);

//calculate the average normals
for( irr::u32 i =0; i < 4; i++)
{

final_norm += norms[i];
final_norm.normalize();

}

// assign the average to each normal
for( irr::u32 i =0; i < 4; i++)
{

norms[i] = final_norm;

}



[christianclavet]

Ok. Thanks for the info that is really pointing me in the right direction.

After looking at your info I Just tested something and found out that the light rendering take the LOCAL vertex position. The test consisted to move the vertices directly instead of moving the node and the shading became a lot nicer. Still I need to use the world and not the local transform for the shader.

I will have to check how set the matrix properly for the shader... Once that done, I will try to blend the vectice normals so it will look smoother.

[mongoose7]

since there are four vertices overlapping at each tile "corner"

try this, I use it for a physics character controller but it should do the same

Code: Select all


irr::core::vector3df* norms;
irr::core::vector3df final_norm(0.0f,0.0f,0.0f);

//calculate the average normals
for( irr::u32 i =0; i < 4; i++)
{

final_norm += norms[i];
final_norm.normalize();

}

// assign the average to each normal
for( irr::u32 i =0; i < 4; i++)
{

norms[i] = final_norm;

}



I think the normalize should be outside the loop.

[HerrAlmanack]

I think the normalize should be outside the loop.

well it DOES make a difference, so far for all my character dynamics applications i have gone with the top panel option, but maybe clavet should decide which is the one required for his application.



EDIT: with a little trig it should be easy to get the bottom panels final angle relative to a horiz. line.

[mongoose7]

The normals are all in the same direction? You can replace the loop with a constant!

[HerrAlmanack]

The normals are all in the same direction? You can replace the loop with a constant!

huh? why in the world would he do that? I'm pretty sure the normals aren't all in the same direction.

[blAaarg]

mongoose7 wrote:
The normals are all in the same direction? You can replace the loop with a constant! Very Happy


huh? why in the world would he do that? Shocked I'm pretty sure the normals aren't all in the same direction.

I think mongoose7 was just trying to make a friendly joke about your specific example. But, I also think he has a point for 2 reasons:

1) Removing the normalizing method from the loop makes it more efficient. Instead of calling normalize() 4 times, you only call it once, and normalize() uses a square root each time which is kind of expensive.

2) Adding the vectors all together at once before dividing (normalizing) them all gives you a true average, rather than giving more "weight" to the normals which get calculated in later.

Using your graphic example above--where 3 normals point in the same direction and have already been normalized--the difference between the 2 methods is maybe 6 or 7 degrees and probably won't look noticeably different. What's more, they will both get all of the duplicated vertices to have the same normal (which should be the solution to the issue), so it should successfully smooth the rendering of the tile edges either way, and in most cases no one would know the difference, if it were that small.