A way to do safe portable asynchronous loading

Discuss about anything related to the Irrlicht Engine, or read announcements about any significant features or usage changes.

A way to do safe portable asynchronous loading

Postby lumirion » Wed Mar 01, 2017 12:51 am

I think this would be a safe way, with minimal modification, to use a list of functors and an openmp parallelized function to asynchronously load meshes.
openmp is included in most compilers. we could test for availability and enable it.

An asyncronus loading process could work like this.
smgr creates an empty scene node of the requested type, and hides it from updating or rendering.

an empty pointer of the correct mesh type is added to the cache to preserve indexes.
(the mesh cache would need a layer of indirection using pointers)

We create functor that takes a pointer to a scene node, a pointer to a mesh, a loader and a path&name to load as arguments. we assign it's arguments and store it in a list.

With all our functors, for the meshes we will load, stored in the list we can parallelize a function to run the functors.

some code showing how we can do this can be seen at http://www.linux-mag.com/id/4609/

if the hardware at runtime is single core with a single thread (or if it is to busy) openmp defaults to synchrony. So, based on hardware capability, and the number of meshes to load, a number of parallel tasks are created and the list divided among them. (each task could have its own list)

Once the functor's mesh is loaded, the pointer in the mesh cache is assigned to it. Then the mesh is assigned to its scene node, and the node is unhidden for smgr.

I don't have time yet to implement and test it but I thought it was interesting.
Posts: 79
Joined: Tue Sep 13, 2011 7:35 am

Re: A way to do safe portable asynchronous loading

Postby devsh » Wed Mar 01, 2017 10:15 am

You dont need to make fake scene nodes, you just need to load an SMesh and the associated textures (without comitting to GPU or driver cache) in a separate thread

then you can either commit both to GPU from the main thread of use IrrlichtBAW and start auxiliary OpenGL contexts and commit to GPU straight from there and pass all that crap to the main thread to only create a scenenode
We chose to stream mesh data from Multiple OpenGL Contexts in many threads and do the other things, not because they are easy, but because they are hard! - JFK
User avatar
Competition winner
Posts: 1764
Joined: Tue Dec 09, 2008 6:00 pm
Location: UK

Re: A way to do safe portable asynchronous loading

Postby lumirion » Fri Mar 03, 2017 1:20 am

Oh! Yah. You're right. Storing an empty scene node till the data is available does seem pointless. Plus would need some sort of mutex on storage access or else limit it to a single asynchronous loader thread.

By the way thanks for updating your git repo. I have been looking forward to browsing your code. If I get some free time I'll test it out.
Posts: 79
Joined: Tue Sep 13, 2011 7:35 am

Return to Open Discussion and Dev Announcements

Who is online

Users browsing this forum: No registered users and 1 guest