JointMesh

Post those lines of code you feel like sharing or find what you require for your project here; or simply use them as tutorials.
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

JointMesh

Post by JVr »

Well after some problems with SkinnedMesh [well it does not mean that SkinMesh is completly bad - it's working well as you know but..] :roll:
(not aligning automatically skin to Identity Matrix, a bit confusing code connected with lot of design work-arounds, some algos suboptimal, not ideal way to use animation from other models, and problematic way to implement HWSkinning in future).

So a made a fork of CSkinMesh called CJointMesh with iterface - IJointMesh. (yes they can be there together so loader writer may choose which of them to use)

It employ:
  • much better algorithms - clearly understandable, well commented & faster
  • improved interface (something that was planned to be done with skinmesh interface in future) - accessible only what is necessary for loader, foolproof, not dependent on actual implementation - (SkinMesh could use this interface too)
  • Different approach for using animation from other models - it is possible to use animations from more that one model and combine them all together
  • Conversion to StaticMesh - make StaticMesh from models without animation or in the middle of animation.
  • Preparation for HW skinning - skinning is independent of animation (all data necessary to do it are stored inside MeshBuffer so it can be done outside JointMesh) and it is possible to change storage of weights without changing interface
so.. in result it should consume equal memory and be faster simpler to understand - and "loader writer friendly" (not letting them to screw anything).

It sould not be so hard to add HW skinning now...
(there is necessary to do some changes in meshbuffers and drivers and I'd prefer if some irrlicht developers whould do it in close future (as it's necessary to add weights and matrix indices to vertex I guess)

Scene node should support transitions (with JointMesh as it do with SkinMesh) and there is CJointSceneNode that makes it possible to attach other nodes to a joint of JointMesh (through new interface method getJointNode(const core::stringc& jointName) - this one works for bouth skinmesh and jointmesh ).

Code "is as it is".. :twisted:

I hope irrlicht woud use parts of this code in future to improve (update) SkinMesh or... use IJointMesh interface and change (or not change) JointMesh and use this new approach for model animation and skinning

(Its expected to work with trunk 1.8+ code and should work with 1.7 too was not checking older versions)

Oki so files that you can actually download:

JointMesh.patch:
Diff against current trunk (1.8.SVN-rev 4054) - Contains ONLY diff for changed files in trunk. (all new files are few lines down)
http://www.mediafire.com/?oqk61dh5b16byew or direct http://www.mediafire.com/file/oqk61dh5b ... .patch.zip
(it's minimal set of chages - may need manual patching as I can't make a patch for every revision - I'll do final patch once 1.8.x is out)

JointMesh Files + loaders - Contains all new files + B3D and X loader that loads to JointMesh
http://www.mediafire.com/?9k4424nyeyjq1cf or direct http://www.mediafire.com/file/9k4424nye ... Joints.zip

This is all you need - note - loaders may not work well coz I did not design them and it was necessary to do some changes in them without complete understanding how they work..

This is version actual to 27.1.2012 (should be latest).
Limitations & Changes:
  • Added one more experimental (may be changed in near future) method loader_addMeshBuffer() that allow adding external (allocated) buffers [for example from other meshes etc..].
  • Bugfix around memory leaks while multimple setMesh is done.
  • Bugfix around animation counting on finalize when animation mapping is used.
  • Some minor bugfixes including transition that was not working well (now it works well and it's tested!!).
  • Fix to enable context in CAnimatedMesh (was not enabled be4).
  • Current implementation support max 4 weights per vertex! (if there is more weights -theese with lowest strength gonna be discarded).
  • Updating BoundingBox for skinned meshes acording actual vertex position was disabled (initial state bounding box is used).
  • Fixed some bugs around AddAnimation reference counting - it should grab and drop them well now.
  • Buffers use only pointer to Matrix to update only pointer not full matrices -> lesser CPU usage.
  • GlobalAnimatedMatrices are now stored inside mesh (joints) to help doing animation without context.
And even if I do not plan to change it much in future I may find some bugs and may do some bugfix updates.
Code is currently in testing stage there may be some more methods added and some more bugfixes
Last edited by JVr on Fri Jan 27, 2012 11:27 am, edited 5 times in total.
RdR
Competition winner
Posts: 273
Joined: Tue Mar 29, 2011 2:58 pm
Contact:

Re: JointMesh

Post by RdR »

Looks good, hope one of the developers has the time to consider this for 1.8.
Do you have any comparison results between the current and your algorithms?
If so, how did you compare them? (Test code)
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: JointMesh

Post by CuteAlien »

Some more info what you did there would really be helpful. Also examples. Unfortunately Luke who wrote the current animation system (at least I think he did, that was before my time) is currently not so often here (he still is sometimes found in chat).
I noticed you modified several existing files - it helps to create a patch in that case so it's easier to see what you changed. And you can post patches to the patch-tracker - that helps us not to forget it.
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Re: JointMesh

Post by christianclavet »

Humm.. This look really interesting. Does you new command to add animation to the mesh will affect all instances of that mesh or can be applied to a specific instance?
The current "use animation from" command that Irrlicht does use affect all the meshes instances. It still useful but you have to use a reference mesh with ALL of the animations inside it, then you can use theses animation on other models that are only skinned.

From what I understand, you can load and append animation from another model to a model. (I suspect that it will affect all instances of the model after that)
This could be more useful for the artists as they could create all of their animation separate and add them with code inside the final model.

Currently I use a tool to merge all of the animation inside a single model. (Fragmotion)
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

Oki to make a tests I needed some Loader able to load Joint and Skin Mesh so I made a modification to X loader:

It's nothing special there would be need to change it much more.. but for my tests this was enuff so... (for example)
http://www.mediafire.com/file/fi3bui4rt ... CXMesh.zip (this uses CJointMesh)
http://www.mediafire.com/?fi3bui4rtx6di4r

Than I needed some piece of code so I modified example 01 - to load Dwarf.x -> http://www.mediafire.com/?q74m1jg6x2x9dp1 (it couts FPS)

Dwarf.X is skinned mesh so impact of improved skinning algo might be visible a bit.

I got results (on my notebook with "Mesa DRI Intel(R) 945GM"):
(I let the example to run some minutes 3-10)
SkinnedMesh
averageFPS 41.181692
averageFPS 42.781089
JointMesh
averageFPS 42.756496
averageFPS 42.812795

Ofc it's nothing hardcore... (the main impact on FPS is not in code of irrlicht but in code of render library and on HW of graphic card)
But it's improvement. And you can see changed interface made it possible to make loader code smaller.

You can make your own tests on DirectX but I'm programming on Linux only so.. (I could test it only on OpenGL maybe burning video but..)
Radikalizm
Posts: 1215
Joined: Tue Jan 09, 2007 7:03 pm
Location: Leuven, Belgium

Re: JointMesh

Post by Radikalizm »

FPS is not a really good unit to measure these things in, do you have any profiling data expressed in the amount of ms it takes to run only your algo against the current one? That would give a much clearer picture of the actual impact your algo has.

It might also be interesting to profile your code for loading animated meshes
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

Full diff file is not soo big - 70kb
http://www.mediafire.com/?f9mnan7b6aamdpo or stright
http://www.mediafire.com/file/f9mnan7b6 ... h_diff.zip
Diff comments why etc..:

/include/CMeshBuffer.h (samall change to allo toSmesh() method to work may be skipped (toSmesh would not work) or method toSmesh() changed)

/include/IAnimatedMesh.h (add EAMT_JOINT)

/include/IAnimatedMeshSceneNode.h (add new generic method virtual ISceneNode* getJointNode(const core::stringc& jointName) )

/include/IJointMesh.h (new file)

/include/IJointSceneNode.h (new file)

/include/irrlicht.h (add include of IJointMesh)

/include/ISceneManager.h (add createJointMesh() method)

/include/WSSkinMeshBuffer.h (new SSkinMeshBuffer.h with weights)

/source/CAnimatedMeshSceneNode.cpp (some changes to introduce possible transition with Joint mesh but..
necessary change is to render EAMT_JOINT buffers same way as EAMT_SKINNED ones.)

/source/CAnimatedMeshSceneNode.h (getJointNode added + support for transition)

/source/CJointMesh.cpp (new file)

/source/CJointMesh.h (new file)

/source/CJointSceneNode.cpp CJointSceneNode.h (new files)

/source/CSceneManager.cpp CSceneManager.h (add createJointMesh)

/source/Makefile - add new files to compile ofc..
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

christianclavet wrote:Humm.. This look really interesting...
As to this:

Once you load a model resulting to IJointMesh you are able to addAnimation (append or replace - depends on mapping offset) from different one. It's done by "projection" of Joints.

Theres a TimeMap that maps frame ranges to diferent CJointMeshes and once program need animation for some frame..
The lookup to this map assign relevant CJointMesh and than there's a search for joint with same name and an animation is taken from the (distant) joint of that (distant) CJointMesh. (wow that would need a picture to descripbe it precisly).

- SkinMesh use somethig I'd call direct mapping that just use pointer to animation you set by UseAnimationFrom (or it points to it's own animation). On the other had JointMesh use indirect mapping while linking everything using Names and IrrMap is used to search for them. The mapping at JointMesh is not dependent on number of mappings. All animations are mapped exactly once except with model own animations that do not need to be mapped at all ofc.

+ of this is that it's memory effective and allow inherit animation from a lot of differnet meshes a combine them together in one mesh.

This (addAnimation) is limited only to JointMesh not to CAnimatedMesh ofc. So once you create a IJointMesh and you make a lot of instances making it CAnimatedMesh - adding animation to IJointMesh (base) will affect all instances of CAnimatedMesh ofc. But if you clone it somehow (there's no copy consturctor.. so it's not possible only by loading one more model) you may have two same looking models with different set of animations.

Is it clear? (Prety abstract problem and although I've done FCE.. I'm not a native speaker. So it's difficult to describe it well)

Well there is a lot of other possiblities - to do mapping faster or add animation without mapping just through adding keys etc.. but theese approaches whould consume more memory soo I implemented this one. (this one was ideal to solve my problem) :)

Ask anything that is not clear about it - I'll try to describe it better.
Last edited by JVr on Thu Dec 15, 2011 9:48 pm, edited 2 times in total.
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

Radikalizm wrote:FPS is not a really good unit to measure
Well yes but the problem is that skinning (old) algo is spread over animation and skinning so I may try to compare
IAnimatedMesh::getMesh( ) but it's not actually used in SkinMesh

I may try to invoke comparsion of ISkinnedMesh::getMesh( ) and IJointMesh::getMesh( ) but well I'll try it later... :wink:

Update:
Result ot test lead to some tweaking of my code that would take some time :)
fmx

Re: JointMesh

Post by fmx »

I looked through your implementation and I noticed a few things you need to be aware of:

currently software-skinning is a hack (not only in your code but also in the original) because what irrlicht requires is weights stored within actual vertex structures and not in hijacked components or seperate arrays, that is the standard way to get HW and SW skinning done these days.

Luke's original code for determining current and next (key)frames was messy, your approach seems a lot more efficient

The CPU bottleneck in both the original and your implementation, is the recursive function to build joint matrices for animation.
If that can be streamlined and made faster even a little, it should lead to much better framerates

I've no idea how well it integrates with irrlicht as a whole, only the devs and experienced irrlicht users can comment on that :wink:
hybrid
Admin
Posts: 14143
Joined: Wed Apr 19, 2006 9:20 pm
Location: Oldenburg(Oldb), Germany
Contact:

Re: JointMesh

Post by hybrid »

first question is why you need another dummy scene node (joint scene node). Can't you use the bone scene node that is already available in Irrlicht to pass your joints from the mesh to the user?
Next thing I'd need is the patch for the .x or .b3d loader to use this mesh type instead of the skinned mesh one. Or any other example which makes use of the full structure of the mesh.
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

OKi finally I'm back with some results for you so:

CBoneSceneNode - dummy scene node for joints - there is a problem in connection of scene node and scene that is for SKinned Mesh completly "inside" CAnimated Mesh so once the "parent" mesh is changed all changes a transmited to ALL children and all childern are actually created once you need only one. It's done that way and much more original bone scene node is somehow sometimes able to transmit change that is state to it to SkinnedMesh.

CJointScene node is actually connected to CJoint and is read only -> so it is not possible to affect CJointMesh with it! IT does not support animators and updates if it's position might be done without help of CAnimatedMesh (but are done here too ofc to support more possible instances etc..). I might use kind of BoneNode to work similar but Bone node store some unnecessary information and has some strange methods... (so making new scene node type was much better solution than let some methods do nothing and let some "dead" data members to take more memory..)

Oki CXMesh loader and CB3D loader are exactly the onse i converted to support CJointMesh - http://www.mediafire.com/?zoai6clsvre0v1a http://www.mediafire.com/file/zoai6clsv ... oaders.zip

To the skinning algo used - original algorithm (SkinMesh) used some (for me) strange hybird using weighting by Joints. That means that weights were connected to joints and were computied byJoint add. CJointMesh uses much more common verison of connection to buffer. Thats why there is WSSkinBuffer. This means that once you sent buffer to render there is (almost) all renderer need to do HW skinning (except pointer to skin matrices that is not so hard to add in current solution).
Resolution of this is taht skinning is done by Buffer -> "All stored vertexes" not by Joint -> "All affected vertexes".

FEW notes to HW skinning in irrlicht:
I was trying to do it ofc however there is a problem with attributes. I need to set "attribute"s - "weights" and "matrice indexes" for every vertex (theese are values that are once per buffer set). Than I ofc need pointer to skinning matrices that are changing every frame.. "uniform" type may be set by callback but I dunno how :twisted: To make hw skinnig working I purpose using type of SkinnigBuffer that will handled differently in lowend graphic rending (at driver code).

While I was tweaking code now (two days lol) I added 3x4 matrices (often used for skinning) theese are much more effective considering memory and lead ofc to less math operations so that improved code speed a bit too (but theese may be used in skinnng mesh too ofc.)

Well I've changed an interface a bit removing completly CJoint from library users. (they don't need to know how data are stored soo rather hide anything that may change again in future) And removing used mesh buffer type from users coz that would change a bit soon I guess :P with adition of hw skinning.

While I was tweaking and profiling code I made some sagnificant changes so there is code that I'm speaking about:
http://www.mediafire.com/?vq08vgrg9yv5bah or stright
http://www.mediafire.com/file/vq08vgrg9 ... Joints.zip
IT should contain all files that were changed if not... I'll sent the rest..

So testing results to test it I'was not so sure but I used this peace of code:
(I used my notebook Core 2 Duo with x64 Linux, and Irrlicht compiled "make NDEBUG=1")

Code: Select all

 
IAnimatedMesh* mesh = smgr->getMesh("../../media/ninja.b3d");
u32 frameN = mesh->getFrameCount();
 
for(int j = 0; j < 400; j++)
        for(int i = 0; i < frameN; i++)
                mesh->getMesh(i);
 
This model have skinning so it employ bouth animation and skinnig part of code. This test is testing only one possible instance if you have more instances of one model the animation will go much differently coz animation hinting cache may work for CSkinMesh but is not supported now for CJointMesh yet but to see the potential.. use this.

first time was common Callgrind (its part of awesome code analytic tool valgrind)
Ofc it does nice data files continaing data about differnet methods etc once debug is set.. but debug was off so... just:

SkinMesh:
==8162== Callgrind, a call-graph generating cache profiler
==8162== Copyright (C) 2002-2010, and GNU GPL'd, by Josef Weidendorfer et al.
==8162== Using Valgrind-3.6.1-Debian and LibVEX; rerun with -h for copyright info
==8162== Command: ./01.CSkinMesh
==8162==
==8162== For interactive control, run 'callgrind_control -h'.
Irrlicht Engine version 1.8.0-alpha
Linux 3.0.0-13-generic #22-Ubuntu SMP Wed Nov 2 13:27:26 UTC 2011 x86_64
Using plain X visual
Loaded texture: /home/jaromir/Dev/irrlicht/irrlicht/media/nskinbl.jpg
Loaded mesh: ../../media/ninja.b3d
==8162==
==8162== Events : Ir
==8162== Collected : 20026697644
==8162==
==8162== I refs: 20,026,697,644

JointMesh:

==17157== Callgrind, a call-graph generating cache profiler
==17157== Copyright (C) 2002-2010, and GNU GPL'd, by Josef Weidendorfer et al.
==17157== Using Valgrind-3.6.1-Debian and LibVEX; rerun with -h for copyright info
==17157== Command: ./01.CJointMesh
==17157==
==17157== For interactive control, run 'callgrind_control -h'.
Irrlicht Engine version 1.8.0-alpha
Linux 3.0.0-13-generic #22-Ubuntu SMP Wed Nov 2 13:27:26 UTC 2011 x86_64
Using plain X visual
Loaded texture: /home/jaromir/Dev/irrlicht/irrlicht/media/nskinbl.jpg
Loaded mesh: ../../media/ninja.b3d
==17157==
==17157== Events : Ir
==17157== Collected : 15267035858
==17157==
==17157== I refs: 15,267,035,858


Oki callgrind is able to cout number of instructions used so this means that Skin mesh use 20K instruction and JointMesh only 15K to do this.
Ofc this does not mean that JointMesh is better - theese 15k instructions may be much slower than 20k of SkinMesh and the cache speed is not mantioned here but there is visible the first difference.

Second test - was real time test:
I was just "time" program:

"time ./CSkinMesh"
real 0m5.833s
user 0m5.808s
sys 0m0.004s

real 0m5.830s
user 0m5.792s
sys 0m0.016s

real 0m5.861s
user 0m5.776s
sys 0m0.052s

"time ./CJointMesh"

real 0m5.168s
user 0m5.132s
sys 0m0.008s

real 0m5.168s
user 0m5.136s
sys 0m0.008s

real 0m5.188s
user 0m5.128s
sys 0m0.044s

This time is ofc much more accurate infroming us about code performance, however driver initialization and mesh loading affect it a bit.

Last test was MemCheck test (Valgrind again).

CSkinMesh

==17342== total heap usage: 3,699 allocs, 2,353 frees, 4,716,952 bytes allocated

CJointMesh

==17060== total heap usage: 27,778 allocs, 25,524 frees, 5,001,037 bytes allocated

The increased memory consumption of CJointMesh may be caused by different model loader (I just made i worknig so it may be bugging etc..) but much more probable case of the storage approach for weights theese are in to two dimensional array that may allocate more than necessary. ( I'll change this array soon I'm thniking what to use :oops: )

But I can say that CJointMesh should use less memory in future.

That's all -> to be continued....
Radikalizm
Posts: 1215
Joined: Tue Jan 09, 2007 7:03 pm
Location: Leuven, Belgium

Re: JointMesh

Post by Radikalizm »

Those are numbers we can work with :D

How much optimization do you think can be applied to this? You're almost getting a 1 ms speed increase which can result in a significant speed increase in crowded scenes, but even more would be better of course

I'd be interested to see where you could go with this, especially since we're working on a native irrlicht animated mesh file format
JVr
Posts: 28
Joined: Wed Oct 19, 2011 2:32 pm

Re: JointMesh

Post by JVr »

Radikalizm wrote:Those are numbers we can work with :D

How much optimization do you think can be applied to this? You're almost getting a 1 ms speed increase which can result in a significant speed increase in crowded scenes, but even more would be better of course

I'd be interested to see where you could go with this, especially since we're working on a native irrlicht animated mesh file format
I've got CJointMesh times:

real 0m4.744s
user 0m4.712s
sys 0m0.004s

So there might be about 20% improvement for skinned models it may be lower for only animated models but there is still a way to improve speed of animation code I hope :)

Now I changed storage of weights one dimensional array (second dimense is fixed normal vector of 4 elements [so maximum weights per vertex is resticted to 4] - that improved access and caching i guess). I'll post that code soon once I test it a bit more. I've lowered memor consumption too but only by 100k so there's some work on it I guess..

I was thinking about storing weights with vertexes doing some kind of extended vertex class (extension of S3DVertex) however that would be pretty big change and I don't wana do it alone. That should lead to better caching.
Last edited by JVr on Thu Dec 01, 2011 4:39 pm, edited 1 time in total.
fmx

Re: JointMesh

Post by fmx »

I'm trying to get my head around where the processing speed gains are coming from, because usually uploading attributes per vertex (weights and IDs) to a shader is faster as far as rendering goes so your method is likely to backfire as soon as you try GPU skinning, even though currently it seems faster with SW skinning

Can you create two simple "real-time timing" tests to compare the performance between current irrlicht and your new JointMesh methods:
1. Animate the joint heirarchy only, do NOT perform any skinning
2. Skin the model to the joints only, do NOT animate the joints

In both cases you should leave everything else the same: same scene, same model, same approach for rendering.
The times are likely to be very very similar, it might help you made the test scene a little more complex, for example using two or three animating models instead of only one

That should tell us what your algos are doing right (or wrong) and what changes might need to be made for better GPU skinning with irrlicht
Post Reply