Updating Tangents and Binormals for Skinned and Morphed Anim

Post those lines of code you feel like sharing or find what you require for your project here; or simply use them as tutorials.

Updating Tangents and Binormals for Skinned and Morphed Anim

Postby Vectrotek » Wed Sep 09, 2015 10:11 pm

C++

When animating a mesh by "Morphing" or "Skeletal Animation" such as "*.md3", "*.x" and "*.b3d" using "Shaders" for rendering we can improve the final render if we "Cyclically Update" the "Tangents" and "Binormals"..
We presume that our meshes are, among others, textured with a "NORMAL MAP" used by the "Shader" (cg, hlsl, or glsl etc) in calculating diffuse and specular.
We also have one or more lights used by the shader.

This is one way we could go about it:

cpp Code: Select all
 
 
 // Update TANGENTS & BINORMALS at every frame for a skinned animation..
   
 // We dont want to do this for static meshes like levels etc..
 // We also dont want to do it for Rotating, Scaled and translated meshes..(we can however, as a bonus, scale, rotate and translate these)
 // Only for animated skinned and morph based meshes..
 // This is loose code that works. If anyone can improve it for the engine itself that would be great..
 // You'll probably ID possible improvements immediately!
 
 // At every N'th Frame we loop through all the vertices..
 // 1. In the loop we Access the VERTEX of POINT A of the "INDEXED TRIANGLE"..
 // 2. We interrogate the "OTHER TWO" VERTICES (which thankfully do change at each frame) for their Positions, Normals, and UV Coords to
 //    Genertate a "BRAND NEW" (animated) TANGENT and BINORMAL. (We may want to calculate the the "Binormal" in the SHADER to save time)
 // 3. We REWRITE the Tangent and Binormal for our SELECTED TRIANGLE POINT.
 // 4. We DO THE SAME for POINTS B and C..
 //
 
 //  GENERATE "LIVING" TANGENTS & BINBORMALS
 //  REMEMBER!
 //  WE NEED "LOOP THROUGH ALL MESHES" THAT ARE TYPE "Skinned"
 //  WE NEED "LOOP THROUGH ALL ITS BUFFERS"
 //  WE NEED "LOOP THROUGH ALL THOSE BUFFER VERTICES"
 // Possible types of (animated) meshes.
 // Enumerator:
 // 1  EAMT_MD2            Quake 2 MD2 model file..
 // 2  EAMT_MD3            Quake 3 MD3 model file..
 // 10 EAMT_MDL_HALFLIFE   Halflife MDL model file..
 // Below is what an item type must be for it to qualify for Tangent Updates..
 // 11 EAMT_SKINNED        generic skinned mesh "*.x" "*.b3d" etc.. (see Morphed too!)
 //                                                                              
 // We want to change tangents for skinned meshes only so we must determine which ones are "Skinned"..
 // This may change if we add and remove meshes during runtime..
 
 // Added this for clarity..
 IMeshCache* TheSceneMeshCache; // Not to be assigned cyclically..
 irr::scene::IMeshBuffer* CurrMeshBuffPtr ;  // We want to assign this cyclically..
 video::S3DVertexTangents* CurrTanVertsPtr ;
 IAnimatedMesh* AcquiredAnimeshByIndex;  // This changes in the loop!
 IMesh*  AcquiredImesh ;
 SceneMeshCount = TheSceneManager ->getMeshCache()->getMeshCount();
 
 //                                                                              
 if (TanUpdStatus == true) // We "set" this bool in our App..
  {                
   //  SceneMeshCount
   for (u32 SMeshI = 0; SMeshI < SceneMeshCount; SMeshI++)  // There amy be other ways to iterate through all your items..
    {// start loop "SMESHI"    
     u16 TheMeshType = TheSceneMeshCache->getMeshByIndex(SMeshI)->getMeshType();
     if (TheMeshType == EAMT_SKINNED)   // EAMT_SKINNED (or morphed ?)..
      {// -start- Only "Skinned" meshes..
       AcquiredAnimeshByIndex = TheSceneMeshCache->getMeshByIndex(SMeshI);
       AcquiredImesh = AcquiredAnimeshByIndex ;
       u32 TheMBufferCount = AcquiredImesh->getMeshBufferCount();
       for (u32 MBuffI = 0 ; MBuffI < TheMBufferCount ; MBuffI++)
        {// start Buffer Loop
         CurrMeshBuffPtr = AcquiredImesh->getMeshBuffer(0);  // WE MUST ALSO LOOP BUFFERS..
         CurrTanVertsPtr  = (video::S3DVertexTangents*)CurrMeshBuffPtr->getVertices(); // Many Buffers for Many Meshes..
         u16* TheINDEXPtr =    TheSceneMeshCache->getMeshByIndex(SMeshI)->getMeshBuffer(0)->getIndices();
         u32 TheIndexCount = (u32)TheSceneMeshCache->getMeshByIndex(SMeshI)->getMeshBuffer(0)->getIndexCount();
         for (u32 IndexII = 0; IndexII < TheIndexCount; IndexII+=3) // Note "+=3" in For loop..
          {// start Faces..  Get all three of our triangle vertices..
            VertexTriA = CurrTanVertsPtr[TheINDEXPtr[IndexII]];
            VertexTriB = CurrTanVertsPtr[TheINDEXPtr[IndexII+1]];
            VertexTriC = CurrTanVertsPtr[TheINDEXPtr[IndexII+2]];
            // We have the animated normals as modelled, so we dont really want to mess with these
            // because during the modelling phase we created "Sharp Edges" (important concept)
            // which play a large role in the final rendering..
            // We could offcourse, when we optimise this, look at how Irrlicht generated these
            // Tangents and Binormals internally (seemingly once, at model loading moment), but for now we'll do it ourselves..
 
            // Here we get the THREE POINTS XYZ Positions for the TRIANGLE..
            f32 TAX = VertexTriA.Pos.X;   f32 TAY = VertexTriA.Pos.Y; f32 TAZ = VertexTriA.Pos.Z;
            f32 TBX = VertexTriB.Pos.X;   f32 TBY = VertexTriB.Pos.Y; f32 TBZ = VertexTriB.Pos.Z;
            f32 TCX = VertexTriC.Pos.X;   f32 TCY = VertexTriC.Pos.Y; f32 TCZ = VertexTriC.Pos.Z;
            // Here we get the UV Coordinates for each of the three Points.
            f32 TAU = VertexTriA.TCoords.X; f32 TAV = VertexTriA.TCoords.Y;
            f32 TBU = VertexTriB.TCoords.X; f32 TBV = VertexTriB.TCoords.Y;
            f32 TCU = VertexTriC.TCoords.X; f32 TCV = VertexTriC.TCoords.Y;
            // We introduce THREE new "Delta Vectors" which will eventually become "Triangle Edges"..
            // This is a special "recipe" using "Triangle Points" and "UV Coordinates"..
            f32 DV1X = TBX - TAX ; f32 DV1Y = TBU - TAU ; f32 DV1Z = TBV - TAV;
            f32 DV2X = TCX - TAX ; f32 DV2Y = TCU - TAU ; f32 DV2Z = TCV - TAV;
            f32 DV3X = TBY - TAY ; f32 DV3Y = TBU - TAU ; f32 DV3Z = TBV - TAV;
            f32 DV4X = TCY - TAY ; f32 DV4Y = TCU - TAU ; f32 DV4Z = TCV - TAV;
            f32 DV5X = TBZ - TAZ ; f32 DV5Y = TBU - TAU ; f32 DV5Z = TBV - TAV;
            f32 DV6X = TCZ - TAZ ; f32 DV6Y = TCU - TAU ; f32 DV6Z = TCV - TAV;
            // Now we introduce THREE "Cross Products". Cross Product A, Cross Product B and Cross Product C.
            // Dont sweat the math, but understand that these Cross Products are needed to get Tangents / Binormals..
            f32 CAX = (DV1Y * DV2Z) - (DV2Y * DV1Z); // "Subtraction" hence "Delta Vectors" above..
            f32 CAY = (DV1Z * DV2X) - (DV2Z * DV1X);
            f32 CAZ = (DV1X * DV2Y) - (DV2X * DV1Y);
            f32 CBX = (DV3Y * DV4Z) - (DV4Y * DV3Z);
            f32 CBY = (DV3Z * DV4X) - (DV4Z * DV3X);
            f32 CBZ = (DV3X * DV4Y) - (DV4X * DV3Y);
            f32 CCX = (DV5Y * DV6Z) - (DV6Y * DV5Z);
            f32 CCY = (DV5Z * DV6X) - (DV6Z * DV5X);
            f32 CCZ = (DV5X * DV6Y) - (DV6X * DV5Y);
            // Calculate our TANGENT..
            f32 TanX = (CAY / CAX);  f32 TanY = (CBY / CBX);  f32 TanZ = (CCY / CCX);
            // ..and our BINORMAL..
            f32 BinX = (CAZ / CAX);  f32 BinY = (CBZ / CBX);  f32 BinZ = (CCZ / CCX);
            // Normalise them.. (is this really necessary??)
            // Apparently not! (I took TWO screengrabs of ON and OFF and they look the same..)
            // f32 TanXN = TanX; // / ((TanX * TanX)+(TanY * TanY)+(TanZ * TanZ));
            // f32 TanYN = TanY; // / ((TanX * TanX)+(TanY * TanY)+(TanZ * TanZ));
            // f32 TanZN = TanZ; // / ((TanX * TanX)+(TanY * TanY)+(TanZ * TanZ));
            // Now this "equals" aint needed.. (I'll keep it here just incase)    
            // Try doing the "Binormals" in the shader by Crossing the Normals with the Tangents..
            // The Irrlicht Coders would probably want this working for Direct X aswell before implementing it..
            // Normalisation apparently not needed here aswell..
            // f32 BinXN = BinX; // / ((BinX * BinX)+(BinY * BinY)+(BinZ * BinZ));
            // f32 BinYN = BinY; // / ((BinX * BinX)+(BinY * BinY)+(BinZ * BinZ));
            // f32 BinZN = BinZ; // / ((BinX * BinX)+(BinY * BinY)+(BinZ * BinZ));
            // Just incase our balls were facing the wrong way..
            // swap (TanXN , BinXN);  swap (TanYN , BinYN); swap (TanZN , BinZN);
            // TanXN *= 1.0; TanYN *= 1.0; TanZN *= 1.0;
            // BinXN *= 1.0; BinYN *= 1.0; BinZN *= 1.0;
            // Now we replace the Static Tangents/Binromals with our animated ones..
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Tangent.X = -TanX;  // Remember, if problems that these were +-TanXN (normalised ones)..  
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Tangent.Y = -TanY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Tangent.Z = -TanZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Tangent.X = -TanX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Tangent.Y = -TanY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Tangent.Z = -TanZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Tangent.X = -TanX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Tangent.Y = -TanY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Tangent.Z = -TanZ;
            // Could this be done in the shader??   I tried but no joy.. ..
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Binormal.X = BinX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Binormal.Y = BinY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Binormal.Z = BinZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Binormal.X = BinX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Binormal.Y = BinY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Binormal.Z = BinZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Binormal.X = BinX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Binormal.Y = BinY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Binormal.Z = BinZ;
            // Now we have "Living Tangents and Binormals" getting updated when we want them to..
            // It means that the interaction between light and surfaces will remain "Physically Correct"
            // during any change made to the geometry such as "Skinned Mesh Deformation"..
           }   // End Index Loop..    
         }  // End Buffer Loop
       } // End loop only "Skinned" meshes..
     } // End loop "SMESHI"    
   }  // End conditional Tangents & Binormal Update..            
 // O.K.  
     
   
   


Get a working example and coplete VC 7 C++ projects here: http://s000.tinyupload.com/?file_id=83795380876460510590
Last edited by Vectrotek on Wed Sep 09, 2015 10:30 pm, edited 1 time in total.
User avatar
Vectrotek
Competition winner
 
Posts: 1076
Joined: Sat May 02, 2015 5:05 pm
Location: South Africa

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Wed Sep 09, 2015 10:15 pm

Thanks Irrlicht has been overdue for this.
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby Vectrotek » Wed Sep 09, 2015 10:25 pm

My pleasure! Keep up the good work you guys!
User avatar
Vectrotek
Competition winner
 
Posts: 1076
Joined: Sat May 02, 2015 5:05 pm
Location: South Africa

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Wed Sep 09, 2015 11:03 pm

One question can I take your cpp file above and add it to my project or do I have to update the Irrlicht sources to make use of your change?
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby Vectrotek » Thu Sep 10, 2015 7:25 pm

@ Glitch.. Yes, I understand why you ask the question. It is rather baffling when you try compile cool code and then discover that the person who posted the code
had changed his own Irlricht "*.dll" without including the DLL or instructions on how the LIB was changed. (I could be wrong on that, considering your "patches" etc which I know nothing of yet)

The "Tangent and Bi-Normal" updating snippet , I actually tried to code in such a way that it would compile with the originally shipped Irrlicht 1.8.1 or 1.8.2 "*.dll" and "*.lib" files!
It should work if your Items i.e. "Meshes", their types, their respective Buffers and their Vertices are accessible in the way that my posted code has it..
I think it is pretty standard, but some folks may have other fancy methods for handling their objects.

From the Irrlicht Engine's point of view, I think an optimized version the code would work just fine.
Note that Tangents and Binormals are already calculated in the engine itself, even if done ONLY ONCE with the "Convert to Tangents" functions..
The thing is, that I couldn't call these functions cyclically as they seemed to do other things too.

So.. The short answer is yes, it should work, because it accesses Meshes independently from how you would, in your App.
It should also work, irrespective of Shader Language, whether GLSL, CG, CGFX, FX or HLSL under OpenGL or Direct X.

Just be sure to add it to your App code in such a way that the declared "Variables" (at the start) is at the "right place" and the "Actual Loop" is at the "right place".
If you should "Add" more meshes to your scene during run-time, then the placement of the "Loop" is important.

I tried to make it as robust as possible by having the loop check all the MESHES for "type-skinned" (morph based "*md2" is possible too).
It may be possible that a "Skinned Mesh" is big enough to warrant "Multiple Buffers", so I also had the loop iterate through all the buffers based on "buffer count per mesh".
You'll see that it is a "for" loop, within a "for" loop, within a "for" loop, within a conditional "if".

I hope it works for Irrlicht Users App Code and eventually inside the Irrlicht Engine as well.
It definitely works in the project I had posted.

Keep me posted! Cheers!
User avatar
Vectrotek
Competition winner
 
Posts: 1076
Joined: Sat May 02, 2015 5:05 pm
Location: South Africa

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Thu Sep 10, 2015 8:09 pm

Okay thanks maybe this will help me fix my Tangent space shader issue on an animated mesh.

Thanks The_Glitch
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Wed Jan 13, 2016 9:31 pm

Vectrotek you application for this has far to many extra things, you should just take a simple Irrlicht demo that's provided with the SDK and add your code for tangents and binormals and only that to show properly how to use it with irrlicht, it would help me as well for converting for use with shader-pipeline without digging through all the extra code.
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby Vectrotek » Tue Jan 19, 2016 4:51 pm

Ill see what I can do!
User avatar
Vectrotek
Competition winner
 
Posts: 1076
Joined: Sat May 02, 2015 5:05 pm
Location: South Africa

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Fri Jan 29, 2016 12:52 am

Vectrotek just add it to the default Irrlicht example it has the least bit of code for simplicity.
http://irrlicht.sourceforge.net/docu/example001.html
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby Vectrotek » Fri Feb 12, 2016 7:20 am

Hi! The Glitch!
I did not merge the code with the example as you asked but I did post a FULL PROJECT
that handles both GLSL and HLSL..

It is a stream lined version of previous code with some cool features..

This is the best I could do so I hope it helps!

Here is the PROJECT..(I forgot to convert from *.x to *.b3d so the file is big.. sorry!)

http://s000.tinyupload.com/?file_id=45737425459221471855

Screenshot:
Image
User avatar
Vectrotek
Competition winner
 
Posts: 1076
Joined: Sat May 02, 2015 5:05 pm
Location: South Africa

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Mon Feb 15, 2016 11:27 am

I'll check into it bud thanks.
Did you get my PM about your fog shader?
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby christianclavet » Sun Jun 26, 2016 4:42 am

Wow! Thanks Vectrotek! Just seen your example and you documented the shader! That's really great! I searched for something like this for a long time! Once they fully switch to shaders, I hope this (or and optimized version) or this will get in Irrlicht.
User avatar
christianclavet
 
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby mongoose7 » Sun Jun 26, 2016 7:30 am

It's really not suitable. If you notice, there is only one (meaning four types, of course) texture per model. It doesn't integrate with the model loading system or the animation system. It would have been much better to have patched Irrlicht than to try and work outside it.
mongoose7
 
Posts: 1227
Joined: Wed Apr 06, 2011 12:13 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby The_Glitch » Sun Jun 26, 2016 1:55 pm

Agree with Mongoose7
User avatar
The_Glitch
Competition winner
 
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Updating Tangents and Binormals for Skinned and Morphed

Postby christianclavet » Sun Jun 26, 2016 4:41 pm

Irrlicht need something like this for a long time. There are issues each time we try to have an animated model with diffuse/spec/normal maps. (There even not a build in shader for this.) This is a good example of that it can be done.

I agree that Irrlicht would need a patch, but a lot of time, patches are not accepted (hendu provided patches for hardware instancing that have never been applied and now they are obsolete). If somebody want to have improved features, it's sometimes preferable to "work outside it", if you want to share it, and not give patches that will never be used.

So who will patch Irrlicht to have this "during the year" if the current devs are so busy?! At least this solution can be used NOW, and could serve as a reference to create a patch later when the devs have the time to do it. Or be used as a example in the distribution...

I'm really grateful that vectrotek showed me how to do it, than to wait ages to see this happening in Irrlicht core. Personally, I'm not an engine programmer at all, and I depend on theses contributions to do anything outside "the box".
User avatar
christianclavet
 
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA

Next

Return to Code Snippets

Who is online

Users browsing this forum: No registered users and 1 guest