JointMesh

Post those lines of code you feel like sharing or find what you require for your project here; or simply use them as tutorials.
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

fmx wrote:I'm trying to get my head around where the processing speed gains are coming from, because usually uploading attributes per vertex (weights and IDs) to a shader is faster as far as rendering goes so your method is likely to backfire as soon as you try GPU skinning, even though currently it seems faster with SW skinning
Well actually hw skinning is nice but first thing about shader is that it's "stupid" so it does not support branches much so common hw skinnig algo just compute
v' = v*W1*w1 + v*W2*w2 + v*W3*w3 + v*W4*w4
or something like that..
SW skinnig is done on FPU..
but I can do some branching for example when I know there's only one weight per vertex (and thats pretty common) i can ignore weights competly and do only v' = v*W1 this is really fast especially with 3x4 matrices and SSE3 (only cache slow it a bit) - this is what orginal SkinMesh algo was not doing ofc..
For other vertices with more weights I do normal computation v' = v*(W1*w1 + W2*w2 + ..) but I don't count any weights with strength = 0 that save me some... more FPU operations (comparing to GPU). Original SkinMesh algo was doing it the way v' = v*W1*w1 + v*W2*w2 + ... and was using 4x4 matrices.

But yes GPU is faster (SW skinning means re-sending all skinned buffers to GPU, HWskinnig means only sending there few 3x4 matrices) and we may hope that once (would be cool to 1.8.0 final) it get implemented in irrlicht so we may use SW skinning only for nonaccelerated rending.

One more note 12-14% of skinning process takes recaluclulation of bounding boxes... ( is it necessary to recalculate them? :P ).
fmx wrote: Can you create two simple "real-time timing" tests to compare the performance between current irrlicht and your new JointMesh methods:
1. Animate the joint heirarchy only, do NOT perform any skinning
2. Skin the model to the joints only, do NOT animate the joints

In both cases you should leave everything else the same: same scene, same model, same approach for rendering.
The times are likely to be very very similar, it might help you made the test scene a little more complex, for example using two or three animating models instead of only one

That should tell us what your algos are doing right (or wrong) and what changes might need to be made for better GPU skinning with irrlicht
I may try this but I need to do a support for hinting in instances of CJointMesh in CAnimatedMesh probably... that whould take a time and I must work a bit on optimization of animation algo... I've done there some heuristics that may save computation of matrices during animation process but it's not much tested (i know its working but not sure if there are all frames ok etc..)
fmx

Re: JointMesh

Post by fmx »

JVr wrote: I may try this but I need to do a support for hinting in instances of CJointMesh in CAnimatedMesh probably... that whould take a time and I must work a bit on optimization of animation algo... I've done there some heuristics that may save computation of matrices during animation process but it's not much tested (i know its working but not sure if there are all frames ok etc..)
That isn't the point, the purpose of the tests is to determine what your algos are doing right which irrlicht currently isn't.
Dont assume that the irrlicht devs will use your new JointMesh system in its entirety.

It might be more economical to improve the current irrlicht implementations rather than replacing them entirely at this point in time (unless the irrlicht devs feel otherwise).

But thanks for your contribution so far anyway :D
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

Well I was thinking much more about implementation and testing and I've found some problems that I've of course solved :twisted:.. ok not all but let's see the progress on this piece of code so..

Animated Mesh loaded from file contain:
  • Joints (actually names of bones used later for Mesh organization)
  • MeshBufers (with infromation about connection to joints)
  • Keys (rotation, scale, poslition) of joints
  • Initial 3Dtransformation of joints
  • Weights - buffer joint connection
This is actually "ONLY" stored in JointMesh. Things like LocalAnimation, GlobalAnimation or Hints are NOT anymore stored inside this class. I've created separate structure holdig this information. The reason why I did this... is that theese are "Instance dependent" so every instance have theese unique depending on current state. I called them "JointInstanceContext". (Skin mesh do ugly "export&import" of hints and LocalMatrices to CAnimatedMesh to make SkinMesh instance independent.

That does not mean that the animation of JointMesh is impossible without them... actually it is not necessary to store them - using them however increase animation speed (see the results later)

Second problem - well SkinMesh::animateMesh method does not actually do all the animation work the reason was there (not necessary to explain why here).. but this method does not affect GlobalAnimatedMatrices. And theese are computed later during SkinMesh method. JointMesh::animateMesh do it all together (well skinning matrices are computed there too as well if mesh contain some joints used for skining).

I made some tests but ofc theese are not final just to see the results:
(I've disabled computation of GlobalAnimatedMatrices and SkinMatrices inside JointMesh::animateMesh to make it create LocalAnimatedMatrix only! - so result of work SkinMesh::animateMesh and JointMesh::animateMesh were equal.)

I used this ugly code.... (SkinMeshCode is commented and Context is enabled in this example):

Code: Select all

 
IAnimatedMesh* mesh = smgr->getMesh("../../media/zombie.b3d");
//CSkinnedMesh* SMesh = reinterpret_cast<CSkinnedMesh*>(mesh);
CJointMesh* SMesh = reinterpret_cast<CJointMesh*>(mesh);
CJointMesh::JointInstanceContext ctx;
SMesh->InitContext(&ctx);
 
for(int j = 0; j < 10000; j++)
        for(int i = 59; i < 75; i++)
        {
                SMesh->animateMesh(i, 0.f, &ctx);
                //SMesh->animateMesh(i, 1.f);
        }
 
I've used zombie.b3d from http://www.psionic3d.co.uk/..

SkinMesh time result for this code was around this:
real 0m0.550s
user 0m0.524s
sys 0m0.012s

JointMesh - without context ( SMesh->animateMesh(i, 0.f); )
real 0m0.526s
user 0m0.504s
sys 0m0.004s
(was a bit faster but not much)

JointMesh - with context was fast :P
real 0m0.429s
user 0m0.388s
sys 0m0.016s

Well to see the reason how could this actually happen I'll provide code but - understand this only as code in progress coz there left some old methods that are no longer used (and will be romved) but.. (animation and skinning methods are I guess final. So to see how is it working it should do..)

Code: Select all

[url]http://www.mediafire.com/?8i56i6yqcx4nvrd[/url] or direct [url]http://www.mediafire.com/file/8i56i6yqcx4nvrd/CJoints2.zip[/url]

Now I'm gonna do an adaptation inside scene - changes inside CAnimatedMesh to support this new interface and update CJointSceneNode.

I've found a problem to do less synthetic test - I want to do some kind of real word test making a scene with some models using EDT_NULL driver to avoid visualisation overload. But I've only few models that may be loated by bouth JointMesh and SkinMesh (types .x or .b3d) and theese models are only skinned so if anyone would have a link to some nice animated model [b]without skinning[/b] It would be cool to make non synthetic tests with such model. So plz link some (only DirectX format or Blitz3D Model format).
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

fmx wrote: It might be more economical to improve the current irrlicht implementations rather than replacing them entirely at this point in time (unless the irrlicht devs feel otherwise).
Ofc that this is your choice but I decided to make it as fork coz it was a pretty big problem to change SkinMesh somehow without affecting all code that use it. Making my own fork make it possible to change anything I want without limitations :mrgreen:
fmx

Re: JointMesh

Post by fmx »

Dont ignore the line above what I said :wink:
it really isn't my choice, it depends on what the irrlicht devs prefer
fmx wrote:Dont assume that the irrlicht devs will use your new JointMesh system in its entirety.
Good luck with your implementation, lets see how it progresses :)
Mel
Competition winner
Posts: 2292
Joined: Wed May 07, 2008 11:40 am
Location: Granada, Spain

Re: JointMesh

Post by Mel »

Well actually hw skinning is nice but first thing about shader is that it's "stupid" so it does not support branches much so common hw skinnig algo just compute
v' = v*W1*w1 + v*W2*w2 + v*W3*w3 + v*W4*w4
You can do branching on shaders, at least PS 3.0 allows the conditional execution of code. I do it on my skinning shader so if i only have one vertex influence, i only do skinning with one bone, and end.
"There is nothing truly useless, it always serves as a bad example". Arthur A. Schmitt
hendu
Posts: 2600
Joined: Sat Dec 18, 2010 12:53 pm

Re: JointMesh

Post by hendu »

Even on current hw it's often faster to do N multiplies vs one branch...
fmx

Re: JointMesh

Post by fmx »

Maybe we should have asked you earlier:
what are you system specs JVr?
The numbers are all very nice but it would help if we knew more about the hardware you are currently using for testing.

If possible, have a go at using your JointMesh system with HW skinning and see what happens.

I use same as Mel's approach, a conditional for each influence before applying any multiplications.
Although saying that, hendu has a point: if a weight is zero then its probably faster to multiply it out anyway instead of doing a conditional test on it.

IIRC a conditional in a shader is worth 8 instructions and multilication is worth 3, or something like that.
Probably depends on the shader compiler, I've heard strange stories of conditionals getting entirely ignored in some cases thanks to attempted optimisations going wrong :P
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

fmx wrote:Maybe we should have asked you earlier: what are you system specs
I guess it's somewere up there but I'm programming on my notebook VAIO VGN-SZ460N (its 2GHz core 2 Duo CPU with Intel integrated i945GM) I may use Nvidia graph card too but I'm saving power :) [Linux Mint 12]

Ok hope I've finished development of JointMesh so I'll show you tests sources and I'll do only bugfixes... (it's still not so much tested so there may be a lot of bugs but.. :lol: )

Ok I've made a testing code and I used static optimized (nondebug) library "make -j 5 NDEBUG=1" - to see final use speed an memory consumption results..

I was really trying to make much less synthetic test (unfortunately I'm still missing non-skinned model).
I've created 5 instaces of one mesh (normal ninja stored in media directory)
Every model got different animation and some different speed of animation.
I used "video::EDT_NULL" as driver to remove rending overhead...

Code: Select all

 
       scene::IAnimatedMesh* ninja = smgr->getMesh("../../media/ninja.b3d");
 
        //normal walk
        scene::IAnimatedMeshSceneNode* ninja1 = smgr->addAnimatedMeshSceneNode(ninja);
        if (ninja1)
        {
                ninja1->setMaterialFlag(video::EMF_LIGHTING, false);
                ninja1->setPosition(core::vector3df(0,5,40));
                ninja1->setFrameLoop(0, 13);
                ninja1->setAnimationSpeed(15);
                ninja1->setScale(core::vector3df(2.f,2.f,2.f));
                ninja1->setRotation(core::vector3df(0,-90,0));
        }
 
        //stealth walk
        scene::IAnimatedMeshSceneNode* ninja2 = smgr->addAnimatedMeshSceneNode(ninja);
        if (ninja2)
        {
                ninja2->setMaterialFlag(video::EMF_LIGHTING, false);
                ninja2->setPosition(core::vector3df(10,5,45));
                ninja2->setFrameLoop(14, 29);
                ninja2->setAnimationSpeed(15);
                ninja2->setScale(core::vector3df(2.f,2.f,2.f));
                ninja2->setRotation(core::vector3df(0,-90,0));
        }
 
        //Idle 1
        scene::IAnimatedMeshSceneNode* ninja3 = smgr->addAnimatedMeshSceneNode(ninja);
        if (ninja3)
        {
                ninja3->setMaterialFlag(video::EMF_LIGHTING, false);
                ninja3->setPosition(core::vector3df(-20,-5,35));
                ninja3->setFrameLoop(183, 204);
                ninja3->setAnimationSpeed(10);
                ninja3->setScale(core::vector3df(2.f,2.f,2.f));
                ninja3->setRotation(core::vector3df(0,-90,0));
        }
 
        //backflip
        scene::IAnimatedMeshSceneNode* ninja4 = smgr->addAnimatedMeshSceneNode(ninja);
        if (ninja4)
        {
                ninja4->setMaterialFlag(video::EMF_LIGHTING, false);
                ninja4->setPosition(core::vector3df(20,-5,35));
                ninja4->setFrameLoop(145,157);
                ninja4->setAnimationSpeed(15);
                ninja4->setScale(core::vector3df(2.f,2.f,2.f));
                ninja4->setRotation(core::vector3df(0,-90,0));
        }
 
        const u32 now = device->getTimer()->getTime();
        double FPSsum = 0;
        u32    uFPSCount;
        while (device->run() && (device->getTimer()->getTime() - now) < 10000 )
        {
                driver->beginScene(true, true, video::SColor(255,113,113,133) );
 
                smgr->drawAll(); // draw the 3d scene
                //device->getGUIEnvironment()->drawAll(); // draw the gui environment (the logo)
 
                driver->endScene();
 
                int fps = driver->getFPS();
                FPSsum += fps;
                uFPSCount++;
        }
 
        device->drop();
        printf("Average FPS %f\n", FPSsum/uFPSCount);
 
It runs precisely 10 seconds and resulting average fps is affected mostly by animation of models (but not only by that)
I've made changes in CAnimatedMeshSceneNode to support context so the first test is
JointMesh (with context):
Average FPS 2340.633114
Average FPS 2348.469008
Average FPS 2343.881827

For next test I've removed usage of context inside "CAnimatedMeshSceneNode" so it's still CJointMesh but using less memory
not remembering last state should slow it a bit :arrow: :
Average FPS 2305.463757
Average FPS 2285.555399
Average FPS 2299.062877

To compare it with skinned mesh I've used normal CB3D loader to load data to skin mesh:
So skin mesh results (current solution) are:
Average FPS 2172.909205
Average FPS 2166.693414
Average FPS 2182.192176

I could use valgrind - memcheck to see how the memory was going...
JointMesh (normal with context):

==20915== HEAP SUMMARY:
==20915== in use at exit: 0 bytes in 0 blocks
==20915== total heap usage: 2,887 allocs, 2,887 frees, 1,590,421 bytes allocated
==20915==
==20915== All heap blocks were freed -- no leaks are possible

SkinMesh:

==23037== HEAP SUMMARY:
==23037== in use at exit: 0 bytes in 0 blocks
==23037== total heap usage: 2,702 allocs, 2,702 frees, 1,541,124 bytes allocated
==23037==
==23037== All heap blocks were freed -- no leaks are possible

Joint mesh allocate a bit more memory if that would be bottleneck it could be a bit optimized and memory consumption lowered it depends on a lot of conditions however the consumption is about 50k more on 5 models -> 10k per model. There is used different model loader that may consume a bit more memory too so it might not consume all this memory at runtime but during load time.

Ofcourse removing context competely would allow lower consumtion, but using context is necessary to support animation transitions.

So I guess you want to see files now so http://www.mediafire.com/?shkh91tlvgylcr1 or directly http://www.mediafire.com/file/shkh91tlv ... oints3.zip

I've added all files that are "changed a lot" - files with minor changes can be found in previous post in diff containing all changes. Theese files are modified ones from current trunk 1.8.SVN

And as for HW skinnig as I worte be4 it's not currently possible since irrlicht does not support something called attributes (well I'm not so sure about that but if it does explain me how :evil:). Attribute is the value connected with each vertex (in this case weight and matrix pointer) so it's not possible to add skinnig without some ugly dirty hacks in drivers that I don't wana do.
mongoose7
Posts: 1227
Joined: Wed Apr 06, 2011 12:13 pm

Re: JointMesh

Post by mongoose7 »

Have you actually tried using the OpenGL renderer, say? You have four skinned models sharing the same mesh, so it would be interesting to see if they each have their own animation or if the mesh is only being animated once.

Also, since skinning is the operation that takes all the time, you could load a mesh of, say, 100000 vertices into Blender and attach a bone to it. Or, since you are working in the joint code, I don't know why you don't simply create a joint yourself for some static mesh.

Finally, you register a gain of 10%. I can't see much point in continuing unless you can make a significant gain. Or do you think your code will adapt to hardware skinning?
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

mongoose7 wrote:Have you actually tried using the OpenGL renderer, say?
Well to see how the scene looks like I did, however for tests I used NULL driver! (how would you exect I'd get 2k+ FPS with openGL? :P
mongoose7 wrote: You have four skinned models sharing the same mesh, so it would be interesting to see if they each have their own animation or if the mesh is only being animated once.
If you'd look to code you'd see that every mesh got different animation and some got different speed o animation
mongoose7 wrote: Also, since skinning is the operation that takes all the time, you could load a mesh of, say, 100000 vertices into Blender and attach a bone to it. Or, since you are working in the joint code, I don't know why you don't simply create a joint yourself for some static mesh.

Well I didn't work only on skinning code but on complete structures used for animation and skinning of models so...
mongoose7 wrote:Finally, you register a gain of 10%. I can't see much point in continuing unless you can make a significant gain. Or do you think your code will adapt to hardware skinning?
So to explain it...
In this test there is not using only my code! Gain in my code is around 20% (synthetic tests up there) however during scene rending there is used much more code so final gain is close to 10%. But ofc my code i not only about CPU gain its about concept that can be used for HW skinning - I can tell you that using HW skinning with current SkinnedMesh is almost impossible (because of data storage) and because of bad interface it's impossible to change it.

And one more reason why I was doing this fork was adaptation of "AddAnimation" method that is able to merge animations from different models together..

And finally there is one more method that is not mentioned above - toSMesh (conversion to static mesh) this is for meshes that do not have animation or later (not implemented yet) for mesh that have an animation to convert current (current position skinning) mesh to static mesh (statue) that can be rendered faster and consume less memory..
mongoose7
Posts: 1227
Joined: Wed Apr 06, 2011 12:13 pm

Re: JointMesh

Post by mongoose7 »

OK, but your meshes are so small that it is difficult to see any gain. That is why general scene handling and rendering take up so much time. Particularly if you want to use hardware skinning, it really is worth putting a large (or a large number of) mesh(es) in the scene.
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

mongoose7 wrote:OK, but your meshes are so small that it is difficult to see any gain. That is why general scene handling and rendering take up so much time. Particularly if you want to use hardware skinning, it really is worth putting a large (or a large number of) mesh(es) in the scene.
Give me link to some files I can use for such test plz :)
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Re: JointMesh

Post by christianclavet »

For the test with the hardware skin code I had, I used as a exemple a 10x10 scene of dwarfs (so basically 100 animates dwarfs. The one that come with the Irrlicht examples) and a dynamic light. Perhaps you could do the same, or even add more instances of them, like a scene with 30x30 dwarfs... Just be sure that your camera can look at them all.
mongoose7
Posts: 1227
Joined: Wed Apr 06, 2011 12:13 pm

Re: JointMesh

Post by mongoose7 »

OR

Go to makehuman. Download the code and export a .obj file. It should have, I think, around 200,000 to 400,000 vertices, but the faces may be quads.

If you are A "real man" you will load it directly and apply a joint to it in Irrlicht. Otherwise, load it into Blender, probably 2.49 as there seem to be complaints about exporters in 2.5. Anyway, you should be able to add some bones and export it as .x or .b3d (I think).

(I'm not a Blender user. I usually take an obj from DAZ Studio, create a skeleton and an animation in a text file and load it all into Irrlicht.)
Post Reply