Updated COctreeSceneNode class for Irrlicht 1.7.3

You are an experienced programmer and have a problem with the engine, shaders, or advanced effects? Here you'll get answers.
No questions about C++ programming or topics which are answered in the tutorials!

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Sun Apr 15, 2018 7:23 pm

This change reduces memory allocations afterwards from 2.4GB to 1.2GB and everything works well after cycle testing during hours.
I also free the keeptriangles using clear after the memcpy to protect system memory from running out on big meshes.
The mesh alone used in testing takes 380 MB.
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Sun Apr 15, 2018 8:49 pm

Yeah, the set_used will create the memory again. It kinda does something similar like the change I proposed would have done. Except you add a few more re-allocations, but as long as it's not costing you too much time when creating the octree it will be fine.

You don't need the KeepTriangles.clear() by the way - it won't make a difference.
edit: yeah - might reduce intermediate memory usage somewhat - just no difference in the endresult.
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Sun Apr 15, 2018 9:48 pm

So how would you do it, with greater efficiency?
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Sun Apr 15, 2018 9:56 pm

With the line of code I posted earlier. That will only resize the array once at the end.
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Sun Apr 15, 2018 10:06 pm

Reallocate it first, then memcpy, that's what my code does.
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Sun Apr 15, 2018 10:40 pm

The difference is that my code can be at the end of the function. And only be called once.
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Sun Apr 15, 2018 10:58 pm

But the array becomes massive consuming more memory than needed, and for large meshes even exhausting all available memory.
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Sun Apr 15, 2018 11:05 pm

Also reallocate copies all the old and redundant data very inefficiently, whereas clear and set are much faster.

Look at reallocate:


//! Reallocates the array, make it bigger or smaller.
/** \param new_size New size of array. */
00063 void reallocate(u32 new_size)
{
T* old_data = data;

data = allocator.allocate(new_size); //new T[new_size];
allocated = new_size;

// copy old data
s32 end = used < new_size ? used : new_size;

for (s32 i=0; i<end; ++i)
{
// data[i] = old_data[i];
allocator.construct(&data[i], old_data[i]);
}

// destruct old data
for (u32 j=0; j<used; ++j)
allocator.destruct(&old_data[j]);

if (allocated < used)
used = allocated;

allocator.deallocate(old_data); //delete [] old_data;
}
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Sun Apr 15, 2018 11:32 pm

reallocate shouldn't be slower (and it would only be called once instead of 8 times).
Memory inside constructOctree will indeed get larger in between when it only cleans up at the end instead of after creating each node. Only end-result would be the same.

Anyway - best solution regarding memory - and avoiding all re-allocation for triangles would be to rewrite it to work with a single array. And all nodes just have range-indices into that array. Wouldn't even be that hard to write. Basically needs a sort function returning numbers for the octet they are in (like 0 for hitting several and 1-8 for some of the others). And that recursively again. But as usual the real work here is not in writing the code but in testing (aka - figuring out test-cases that give realistic speed-comparisons for real situations - and mainly of interest is certainly getTriangles speed and not creation and that might change, thought without profiling I wouldn't even guess if the effect would be positive or negative as there are arguments for both).
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Mon Apr 16, 2018 12:03 am

Fine if your meshes are small.

Why create another triangle buffer at all, the indices could index directly into the smesh buffers, just add a word in the indices array to indicate sub buffer.
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Mon Apr 16, 2018 12:22 am

Hm, can't avoid the copy as you need the sorting. One could use sorted indices, but I guess you wouldn't really save much that way.
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Mon Apr 16, 2018 5:44 am

In the copy in iarray, can't parallel_for be used if the array is larger than a few hundred elements, get those other core working?
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Mon Apr 16, 2018 10:31 am

Different topic really :-)
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby robmar » Mon Apr 16, 2018 10:38 am

Of course, just an idea, as would be an easy boost to employ
robmar
 
Posts: 1028
Joined: Sun Aug 14, 2011 11:30 pm

Re: Updated COctreeSceneNode class for Irrlicht 1.7.3

Postby CuteAlien » Mon Apr 16, 2018 11:37 am

Did you ever try if that's really faster? And by try I mean - did you measure it? I didn't so far (I'm not really interested that much in compiler specific stuff usually as that kind of libs tend to get replaced with cross-platform libs). Given how good processors are in processing vectors my first guess would be that this is one of the cases where things would be somewhat tricky to improve easily by processing it parallel. But my knowledge of how multiple cores access shared memory and how caches are handled in that case is somewhat shallow. Guess I should read up on things like hyperthreading some day.
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8533
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

PreviousNext

Return to Advanced Help

Who is online

Users browsing this forum: No registered users and 1 guest