Idea about improving the Loadscene command

Discuss about anything related to the Irrlicht Engine, or read announcements about any significant features or usage changes.
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Idea about improving the Loadscene command

Post by christianclavet »

Hi, I'm having an idea that could help while loading scene files.

I've not yet found where are the methods in the source, someone know where they are (I saw currently 2 methods there in the API docs)

Code: Select all

virtual bool  loadScene (const c8 *filename, ISceneUserDataSerializer *userDataSerializer=0)=0 
  Loads a scene. Note that the current scene is not cleared before. 


Perhaps the idea is already implemented. What is the ISceneUserDataSerializer?

Here is the idea:

Creating new functions to load scenes, but in step (a per node loader with status).

The current problem I face, is that if I have a big scene (And in my project, I'm sure, the levels will take some time to load) we don't have any feedback and it will leave use with the impression the application is "stuck".

So if a command loaded a level but PER NODE and had some feedback info on the next node that could be loaded and the amount loaded (ex: node 5 on 203). We could create some feedback to let the user know the application is still alive. (It will not help if the mesh is very big, but at least it's a start)

The functions could be created like that (should not replace the older ones for keeping compatibility):
loadScene(const c8 *filename, NodeLoader * loader)
Will load the entire xml file in memory and parse the nodes and put it in the new class NodeLoader.

Once it has been retrieved and the NodeLoader has the info we could then do this:
int currentNode()
Retrieve the position of the current loaded node

int lastNode()
Retrieve the position of the last node in the XML

string nextNodeType()
Retrieve the next node type (Occtree, Animated mesh, skybox, etc.)

c8 nextFilename()
Retrieve the next loading node filename (So you could use the file system to check for the filesize), if the file is big, put a message telling "wait"

bool loadNextNode()
Will load the next scene node and give a return value if the node as been successfully loaded

Once the loading is complete, you would need to destroy(delete) the object (NodeLoader)

I think that in the future I could be able to do this, I would surely need some help to clean the code because, I'm still learning how to code properly in C++. I'll still have to check the source for the current functions because all of the needed code is almost there. I think it would need to be double or tripple checked, because such a source done by a begginer like me could be full of memory leaks as I think memory should be allocated by the heap. (To keep the object data)

Could this be possible or there is a much more efficient way to load a scene with ways to get info at each loaded node?
At first I could at least propose this as a feature request for the next IRRlicht release if the idea is good.
Last edited by christianclavet on Wed Jul 30, 2008 3:04 pm, edited 2 times in total.
JP
Posts: 4526
Joined: Tue Sep 13, 2005 2:56 pm
Location: UK
Contact:

Post by JP »

Yeah this would be cool, even fairly simple scenes often take some time to load so a scene in a large game would obviously take a serious amount of time and feedback would be nice.

But then loadScene is blocking so you'd only get the feedback once it's completed unless you used multithreading, which isn't safe anyway. So you can't render whilst loadScene is executing anyway.. so that's a bit of a problem.

One way to solve it could be to pass a render function as an extra parameter to loadScene and then after each node is loaded it calls that render function to update progress on screen.
Image Image Image
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Post by christianclavet »

Hi, JP.

I don't think you'll need some background loading for this. An XML file is fairly small compared to meshes. It shoulnt take too long to load.

If you check what I propose, it will not load the entire scene at once, but node per node. So you can do that on your own (as refresh the screen, fire a bullet from a cannon, until you trigger that: bool loadNextNode()

If background loaders are implemented in the future then you could load the entire scene and put some action on the screen while it's loading. But if that idea if implemented, at least we could create a percent loaded with infos on what is being loaded. While at the moment, we can only write "wait" before loading the entire scene.
JP
Posts: 4526
Joined: Tue Sep 13, 2005 2:56 pm
Location: UK
Contact:

Post by JP »

Ahh yeah i didn't fully read your proposition. It seems like quite a good idea, a bit clunky but should work i think. It's something which you could probably do yourself (without needing to edit the source probably) and then create a patch or something.
Image Image Image
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Post by christianclavet »

Yes, I'm thinking of doing it first as includes files (without any patching) as a prototype. If someone could point me at the current source of loadscene as I don't seem to find it in the IRRlicht source. I need to refer to that to create the new thing. (As I've done with my camera rig, by checking how it was done with the FPS Camera)
sudi
Posts: 1686
Joined: Fri Aug 26, 2005 8:38 pm

Post by sudi »

i have another idea to improve the loadScene command.
maybe it should have option to pass a parent scenenode to the loaded scene. that way u could use the the sceneformat to prepare probs for ur game then load them and have it seperate from the rest of the scene. so u could use it to setup particle system load them and directly have a handle to them. and when passing NULL as parent the scene is loaded like now.
We're programmers. Programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. We're not excited by renovation:tinkering,improving,planting flower beds.
JP
Posts: 4526
Joined: Tue Sep 13, 2005 2:56 pm
Location: UK
Contact:

Post by JP »

i like the idea of having a parent.. that way you could create seperate parts of your overall scene and have them with seperate parents which could then be made visible/invisible as necessary to improve rendering of large scenes.

christian, just open ISceneManager.cpp and use the find function to locate loadScene ;)
Image Image Image
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Post by christianclavet »

Thanks JP. Found it.

Also found the function I'll need to "replace"
here it is:

Code: Select all

/! Loads a scene. Note that the current scene is not cleared before.
bool CSceneManager::loadScene(io::IReadFile* file, ISceneUserDataSerializer* userDataSerializer)
{
	if (!file)
	{
		os::Printer::log("Unable to open scene file", ELL_ERROR);
		return false;
	}

	io::IXMLReader* reader = FileSystem->createXMLReader(file);
	if (!reader)
	{
		os::Printer::log("Scene is not a valid XML file", file->getFileName(), ELL_ERROR);
		return false;
	}

	// for mesh loading, set collada loading attributes

	bool oldColladaSingleMesh = getParameters()->getAttributeAsBool(COLLADA_CREATE_SCENE_INSTANCES);
	getParameters()->setAttribute(COLLADA_CREATE_SCENE_INSTANCES, false);

	// read file

	while(reader->read())
	{
		readSceneNode(reader, 0, userDataSerializer);
	}

	// restore old collada parameters

	getParameters()->setAttribute(COLLADA_CREATE_SCENE_INSTANCES, oldColladaSingleMesh);

	// finish up

	reader->drop();
	return true;

}
There a while loop in there:

Code: Select all

while(reader->read())
	{
		readSceneNode(reader, 0, userDataSerializer);
	}
readSceneNode(reader, 0, userDataSerializer); is perfect. So I would not have to create that, just a way to count the scene nodes. Just have a way to keep in memory the pointer to reader, so we could refresh the screen before each loading node an write something. (ex: loading element X of scene on the gui)

Humm the method is private. Do you think I could create a new class that could inherit the method? (Or create a duplicate of it?)

EDIT: Humm. Looking at the source and I think I found out what is userDataSerializer. This seem to be related to custom properties we could create for nodes. This seem interesting because I could create entities info on specific nodes (animated meshes). Am I wrong about assuming that?

This seem possible even for a noob like me! :lol:
sudi
Posts: 1686
Joined: Fri Aug 26, 2005 8:38 pm

Post by sudi »

christianclavet maybe u should provide a patch that enables the stuff u wants and maybe make that parentoption avaible....then i'm the first who downloads it.....ok could do it myself but right now i'm not allowing myself to programm cause i gotta get ready for my college exams.
We're programmers. Programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. We're not excited by renovation:tinkering,improving,planting flower beds.
full.metal.coder
Posts: 68
Joined: Sat May 10, 2008 11:30 am
Contact:

Post by full.metal.coder »

Actually you got to rewrite creadSceneNode as well since it is not only private but internal (ISceneManager has not such method...). Fortunately this method is not very complicated since all the grunt work is done in the scene node factory. :D

I'm already playing with this for two reasons :
  • * nice loading screen as mentionned earlier
    * just in time physics setup (instead of full scene traversal once it is loaded)
edit : getting around the loading process is a bit trickier than I expected and will probably require a patch...
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Post by christianclavet »

Hi, Sudi.

Sure! :wink:
But this will not be done today! I'm rewriting my level editor, and this level editor will need to load scenes. (I'd like to share the code with fullmetalcoder that is writing the TFK engine so we have a common model to load scenes).

Right now, i'm back in the beginning and this will be worked much, much later. I just had an idea this morning because I was thinking how much long (because of no feedback) it was to load scenes.

Meanwhile, will it change a lot of things if we change that function from private to public?
readSceneNode(reader, 0, userDataSerializer);

Only using it (preparing the data before), will solve most of the problems. I think once that function is public it could be very easy to do this. Unless we can define a new class that inherit that and access it. (is that possible?).
edit: Yes, You're right. We would have to change the 2 headers files to make the change (ISceneManager.h and CSceneManager.h). But if changed, can this affect the rest of the engine?

IScenemanager.h had only the public, while CSceneManager.h has the public and the private (this one seem complete) (I'm still confused how this work, having 2 header files for the same .cpp file)
Last edited by christianclavet on Wed Jul 30, 2008 4:54 pm, edited 1 time in total.
full.metal.coder
Posts: 68
Joined: Sat May 10, 2008 11:30 am
Contact:

Post by full.metal.coder »

Proposed solution : add a new virtual method to ISceneUserDataSerializer :

Code: Select all

void OnCreateNode(ISceneNode *node);
This method is called from CSceneManager::readSceneNode() every time a new scene node is created during the loading process. It makes it possible to do some nice things from this handler as mentionned earlier.

Another important point about feedback : It is possible to make use of the video driver during the scene loading as long as you don't attempt to draw the scene you're filling with data :roll:

In most case 2d rednering will be good enough but I you fancy adding a cute 3d scene (cutscene for instance) it can be done by creating a new scene manager through createNewSceneManager() and using it until the new scene is ready.

The patch itself (6-liner, or 2-liner if you only take actual code lines into account) : http://edyuk.org/misc/scene-loading.patch
christianclavet
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA
Contact:

Post by christianclavet »

Another important point about feedback : It is possible to make use of the video driver during the scene loading as long as you don't attempt to draw the scene you're filling with data
Humm. I think it would be possible to draw the entire scene between each call of this function (if it's made public ...):
void CSceneManager::readSceneNode(io::IXMLReader* reader, ISceneNode* parent, ISceneUserDataSerializer* userDataSerializer)

You mean if it make public, you could do things while this function operate?
full.metal.coder
Posts: 68
Joined: Sat May 10, 2008 11:30 am
Contact:

Post by full.metal.coder »

You can not do things while other things are going on unless you use threads. Using threads to do the loading would probably be a bad idea.Contrary to what people tend to think threads make things slower (unless you run your app on a mutli-core CPU using a system smart enough to dispatch threads properly).

The only problem is that threading is a complicated an platform dependant stuff so you generally don't want to mess with it unless the benefit is huge and your toolkit makes their use easy. Irrlicht is not designed for this (it has no threading support and most of the API is not thread-safe).

Rendering the scene being loaded probably won't crash but it is extremely likely to look ugly. Besides, if you are supposing your scene is slow to load it probably won't be fast to render so you end up slowing the loading process even more which is not what you want.

Thus, sticking with GUI/direct drawing with video driver/an alternate scene manager (possibly temporary or kept for all loading) is the best way to have some visual feedback.

Also, there is no need to make readSceneNode() public. Just subclass ISceneUserDataSerializer, reimp its three methods (out of which only two are useful for loading) and pass it to loadScene().

If you want visual feedback just call IVideoDriver::beginScene() from the OnCreateNode() handler, draw your stuff call IVideoDriver::endScene() and return to loading.

This method does not give you percentages but live feedback which is good enough as the user won't believe that the app is stuck.

If you want percentages you'll have to load the level as XML on your own and traverse it, counting the number of nodes while you do so. Then percentage would be easy to make but I'm not sure they are worth it...
hybrid
Admin
Posts: 14143
Joined: Wed Apr 19, 2006 9:20 pm
Location: Oldenburg(Oldb), Germany
Contact:

Post by hybrid »

I'd once again suggest my async load mechanism for this purpose. Instead of actually loading the scene you'd get back a scene loader object. This one could expose several ways to load a scene, either one full load (maybe even as part of a cast-to-ISceneNode operator, which would make it pretty backwards compatible) and a read-next-chunk method which would load only some part of the file. Since the loader knows how much of the file has been processed, a percentage meter would be possible as well. This doesn't solve the multi-threading issues, so smooth animations might not be possible, but it would allow for arbitrary deferral of scene loading and feedback of the loading status.
Post Reply