ITerrainSceneNode mem-bug??, cant remove it.HELP!!!

If you are a new Irrlicht Engine user, and have a newbie-question, this is the forum for you. You may also post general programming questions here.
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

ITerrainSceneNode mem-bug??, cant remove it.HELP!!!

Post by bonsalty »

It pumps my memory to hell.

I have the following problem:

There is a terrainscenenode, defined as global. After some loop of rendering the terrain has to be removed and updated.

I have made some easy tests and they fail, here it is.

Code: Select all


        .
        .
        scene::ITerrainSceneNode *terrain; 

        // the LOOP comes

	while(device->run())
	if (device->isWindowActive())
	{
	
	
	       // Adding the node

		 terrain = smgr->addTerrainSceneNode(
		"../../media/viz.png",
		0,					// parent node
		-1,					// node id
		core::vector3df(0.f, 5.0f, 0.f),		// position
		core::vector3df(0.f, -90.f, 0.f),		// rotation
		core::vector3df(100.f,15.0f, 100.f),	// scale
		video::SColor ( 255, 255, 255, 255 ),	// vertexColor
		3,					// maxLOD
		scene::ETPS_65,				// patchSize
		0					// smoothFactor
		);

	terrain->setMaterialTexture(0,tex1);
	terrain->setMaterialTexture(1,tex2);
	terrain->setMaterialType(video::EMT_TRANSPARENT_ADD_COLOR);
	terrain->setMaterialType(video::EMT_REFLECTION_2_LAYER);
	terrain->setMaterialFlag(video::EMF_LIGHTING, false);
	

       // now rendering...
	 		
		driver->beginScene(true, true, 0 );
		
			smgr->drawAll();
			env->drawAll();
							
			
		driver->endScene();
	
		
	terrain->remove(); //this is my problem. 

		
	}
The node should be removed from the scene on the next rendering loop.However visually it really disapears, but not from the memory. After a few loops the memory gets filled up.
Putting terrain->remove(); before the rendering process works fine, but I need to remove it after.

I dont get it, what happens. Please help!
Tomi
randomMesh
Posts: 1186
Joined: Fri Dec 29, 2006 12:04 am

Re: ITerrainSceneNode mem-bug??, cant remove it.HELP!!!

Post by randomMesh »

You don't want to add a terrain every frame.
"Whoops..."
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

Post by bonsalty »

Of course not, I made a program that loads certain maps. Its just a test as I mentioned. But after loading and removing 20 maps from the scene the program eats 500 mB memory , its just not normal. So how to solve it, am I wrong or Irrlicht is that kind of buggy?
Tomi
randomMesh
Posts: 1186
Joined: Fri Dec 29, 2006 12:04 am

Post by randomMesh »

bonsalty wrote:Of course not, I made a program that loads certain maps.
But your code does exactly that.
Then why do you post irrelevant code? Not the best way to get help. Maybe a minimal compilable example would be more suitable.
"Whoops..."
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

Post by bonsalty »

Here it is, just watch the Taskmanager :)

Code: Select all

#include <irrlicht.h>
#include <iostream>
#include <AfxWin.h>


using namespace irr;

#ifdef _MSC_VER
#pragma comment(lib, "Irrlicht.lib")
#endif


int main()
{
	// let user select driver type

	video::E_DRIVER_TYPE driverType = video::EDT_DIRECT3D9;

	int i=0;
	switch(i)
	{
		case 1: driverType = video::EDT_DIRECT3D9;break;
		case 2: driverType = video::EDT_DIRECT3D8;break;
		case 3: driverType = video::EDT_OPENGL;   break;
		
	}	

	IrrlichtDevice* device = createDevice(driverType,
			core::dimension2d<s32>(800, 600));

	
	
	video::IVideoDriver* driver = device->getVideoDriver();
	gui::IGUIEnvironment* env = device->getGUIEnvironment();
	driver->setTextureCreationFlag(video::ETCF_ALWAYS_32_BIT, false);
	device->getCursorControl()->setVisible(false);

	scene::ISceneNode* skydome;
	video::ITexture *tex1,*tex2;
	tex1=driver->getTexture("../../media/irrlichtlogo.jpg");
	tex2=driver->getTexture("../../media/irrlichtlogo.jpg");
	scene::ISceneManager* smgr ;
	smgr= device->getSceneManager();
		
	 skydome=smgr->addSkyDomeSceneNode(driver->getTexture("../../media/skydome.jpg"),16,8,0.77f,2.0f);


    scene::ICameraSceneNode* camera=smgr->addCameraSceneNodeFPS(0,100.0f,5.2f);
	camera->setFarValue(52000.0f);

		
	scene::ITerrainSceneNode* water;

	while(device->run())
	if (device->isWindowActive())
	{
	
		
		camera=smgr->addCameraSceneNodeFPS(0,100.0f,5.2f);

		 water = smgr->addTerrainSceneNode(
		"../../media/irrlichtlogo.jpg",
		0,					// parent node
		-1,					// node id
		core::vector3df(0.f, 5.0f, 0.f),		// position
		core::vector3df(0.f, -90.f, 0.f),		// rotation
		core::vector3df(100.f,15.0f, 100.f),	// scale
		video::SColor ( 255, 255, 255, 255 ),	// vertexColor
		3,					// maxLOD
		scene::ETPS_65,				// patchSize
		0					// smoothFactor
		);

	water->setMaterialTexture(0,tex1);
	water->setMaterialTexture(1,tex2);
	water->setMaterialType(video::EMT_TRANSPARENT_ADD_COLOR);
	water->setMaterialType(video::EMT_REFLECTION_2_LAYER);
	water->scaleTexture(25.0f, 1.0f);
	water->setMaterialFlag(video::EMF_LIGHTING, false);
	


	 		
		driver->beginScene(true, true, 0 );
		
			smgr->drawAll();
			env->drawAll();
			
				
			water->remove();
			//smgr->clear();
		driver->endScene();
	
		
		
		
	
		
	}

	device->drop();
	
	return 0;
}
[/code]
Tomi
randomMesh
Posts: 1186
Joined: Fri Dec 29, 2006 12:04 am

Post by randomMesh »

This code adds a terrain (and camera) every frame too. :?

The main loop starts with

Code: Select all

while(device->run())
in case you don't know.
"Whoops..."
CiRiuS2
Posts: 14
Joined: Mon Oct 26, 2009 12:14 am
Location: Spain

Post by CiRiuS2 »

I think here is your solution :wink:

Code: Select all

#include <irrlicht.h> 
#include <iostream> 
#include <AfxWin.h> 


using namespace irr; 

#ifdef _MSC_VER 
#pragma comment(lib, "Irrlicht.lib") 
#endif 


int main() 
{ 
   // let user select driver type 

   video::E_DRIVER_TYPE driverType = video::EDT_DIRECT3D9; 

   int i=0; 
   switch(i) 
   { 
      case 1: driverType = video::EDT_DIRECT3D9;break; 
      case 2: driverType = video::EDT_DIRECT3D8;break; 
      case 3: driverType = video::EDT_OPENGL;   break; 
       
   }    

   IrrlichtDevice* device = createDevice(driverType, 
         core::dimension2d<s32>(800, 600)); 

    
    
   video::IVideoDriver* driver = device->getVideoDriver(); 
   gui::IGUIEnvironment* env = device->getGUIEnvironment(); 
   driver->setTextureCreationFlag(video::ETCF_ALWAYS_32_BIT, false); 
   device->getCursorControl()->setVisible(false); 

   scene::ISceneNode* skydome; 
   video::ITexture *tex1,*tex2; 
   tex1=driver->getTexture("../../media/irrlichtlogo.jpg"); 
   tex2=driver->getTexture("../../media/irrlichtlogo.jpg"); 
   scene::ISceneManager* smgr ; 
   smgr= device->getSceneManager(); 
       
    skydome=smgr->addSkyDomeSceneNode(driver->getTexture("../../media/skydome.jpg"),16,8,0.77f,2.0f); 


    scene::ICameraSceneNode* camera=smgr->addCameraSceneNodeFPS(0,100.0f,5.2f); 
   camera->setFarValue(52000.0f); 

       
   scene::ITerrainSceneNode* water; 

    
    

       water = smgr->addTerrainSceneNode( 
      "../../media/irrlichtlogo.jpg", 
      0,               // parent node 
      -1,               // node id 
      core::vector3df(0.f, 5.0f, 0.f),      // position 
      core::vector3df(0.f, -90.f, 0.f),      // rotation 
      core::vector3df(100.f,15.0f, 100.f),   // scale 
      video::SColor ( 255, 255, 255, 255 ),   // vertexColor 
      3,               // maxLOD 
      scene::ETPS_65,            // patchSize 
      0               // smoothFactor 
      ); 

   water->setMaterialTexture(0,tex1); 
   water->setMaterialTexture(1,tex2); 
   water->setMaterialType(video::EMT_TRANSPARENT_ADD_COLOR); 
   water->setMaterialType(video::EMT_REFLECTION_2_LAYER); 
   water->scaleTexture(25.0f, 1.0f); 
   water->setMaterialFlag(video::EMF_LIGHTING, false); 
    

   while(device->run()) 
   if (device->isWindowActive()) 
   { 


           
      driver->beginScene(true, true, 0 ); 
       
         smgr->drawAll(); 
         env->drawAll(); 
          
             
        // water->remove(); 
         //smgr->clear(); 
      driver->endScene(); 
    
       
       
       
    
       
   } 

   device->drop(); 
    
   return 0; 
} 
hybrid
Admin
Posts: 14143
Joined: Wed Apr 19, 2006 9:20 pm
Location: Oldenburg(Oldb), Germany
Contact:

Post by hybrid »

Well, the actual problem of the first code is not that he adds a terrain every frame. That was just for testing purposes. The problem is that the mesh is not properly removed from the mesh cache. The static mesh is created by the terrain scene node, and it is put into the mesh cache for fast retrieval. But remove() only removes the scene node, not the mesh. Just get the pointer or name from the terrain node and remove it from mesh cache after the remove() call.
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

Post by bonsalty »

Its good to hear there is a solution.
How can I remove the cache, by getting the pointer?
Thanks.
Tomi
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

Post by bonsalty »

Ok I understand what you say,checked it out,but Im basic in irrlicht and
couldnt find the solution..

"Just get the pointer or name from the terrain node"
I thought "terrain" is the pointer refering to the node:

Code: Select all

terrain = smgr->addTerrainSceneNode(.... 
if not, how to fetch the static one?

"and remove it from mesh cache after the remove() call."
Do you mean:

Code: Select all

smgr->getMeshCache()->clear();
the mesh is empty, I guess its for animated loaded meshes like Q3map

I would be very happy if we could solve this problem, since its not properly covered and Im sadly running out of time. :cry: [/code]
Tomi
hybrid
Admin
Posts: 14143
Joined: Wed Apr 19, 2006 9:20 pm
Location: Oldenburg(Oldb), Germany
Contact:

Post by hybrid »

There's indeed no mesh generated which also gets stored in the mesh cache. It only happens for TerrainMesh, not TerrainSceneNode. The terrain scene node uses a statically allocated SMesh, which is ot really correct use, but shouldn't hurt either. So right now I didn't see any problems with the code. I need to investigate this further.
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

Post by bonsalty »

Ok, I removed one topic.
Tomi
bonsalty
Posts: 120
Joined: Thu Dec 10, 2009 1:30 pm
Location: Budapest,Hungary

Post by bonsalty »

If this helps:
I removed the camera declared outside the loop,so no camera. In that case, the memory deallocation works fine for the node, ofcourse nothing visible when drawing. Im just confused what the connection there is. If the static mesh doesnt get removed, then why without camera? It must be the scene node which doesnt drops it, but if that, why is it then visually removed from scene?
The camera doesnt let it remove from mem. Hmm ...
Tomi
vitek
Bug Slayer
Posts: 3919
Joined: Mon Jan 16, 2006 10:52 am
Location: Corvallis, OR

Post by vitek »

The original testcase can be simplified and corrected a bit. The end result is the following...

Code: Select all

#include <irrlicht.h>
using namespace irr;

#ifdef _MSC_VER
#  pragma comment(lib, "Irrlicht.lib")
#endif

int main()
{
  // let user select driver type
  video::E_DRIVER_TYPE driverType = video::EDT_OPENGL;

  IrrlichtDevice* device = createDevice(driverType,
    core::dimension2d<u32>(800, 600));
  if (!device)
    return 1;

  video::IVideoDriver* driver = device->getVideoDriver();
  scene::ISceneManager* smgr = device->getSceneManager();

  scene::ICameraSceneNode* camera = smgr->addCameraSceneNodeFPS(0,100.0f,5.2f);
  camera->setFarValue(52000.0f);

  for (u32 i = 0; i < 500; ++i)
  {
    scene::ISceneNode* node = smgr->addTerrainSceneNode(
      "../../media/irrlichtlogo.jpg");

    if (driver->beginScene(true, true, 0))
    {
      smgr->drawAll();
      driver->endScene();
    }

    node->remove();
  }

  device->drop();

  return 0;
}
I traced it down to the drawMeshBuffer() call in CTerrainSceneNode::render(). As it turns out, a SHWBufferLink is getting allocated for the terrain mesh data and inserted into the HWBufferMap. The entry is never removed from the HWBufferMap.

There are several possible workarounds. The easiest is to remove the SHWBufferLink entry for the terrain node when removing it from the scene. There is a chance that the buffer will be needed later if the remove() call doesn't delete the node, but the SHWBufferLink will be recreated if necessary.

Code: Select all

driver->removeHardwareBuffer(node->getRenderBuffer());
node->remove();
Another option would be to disable VBO for the terrain node render buffer. That would probably be bad for performance. The final option is to call driver->updateAllHardwareBuffers() which will eventually deallocate buffers that have not been used for a while**.

It seems like the the mesh buffers that are VBO enabled need to be sure to remove themselves when they are being destroyed. Since the data is no longer valid, it makes no sense for the VBO entry to exist any longer. Unfortunately, this would require that each mesh buffer would need to keep track of the driver that contains the VBO. This is not a great solution. Another option would be to require all users of IMeshBuffer be sure to remove the SHWBufferLink for the buffers that have been dropped for the last time.

I'm not sure what the best fix is, but it seems that the design of the VBO requires the user to think more about cleaning up resources, and this is going to be problematic and frustrating for users.

Travis

** The updateAllHardwareBuffers() function only cleans up meshes that have not been used more than 20000 calls to that function. That means that periodic calls to that function will fail to unload the VBO data in a reasonable amount of time (1 call per second would require the mesh to be unsed for 5.55 hours). IMO, this method should also take a parameter that is the maximum age of a VBO before it is deleted. There should also be a flag indicating that a VBO should never be removed automatically.
hybrid
Admin
Posts: 14143
Joined: Wed Apr 19, 2006 9:20 pm
Location: Oldenburg(Oldb), Germany
Contact:

Post by hybrid »

updateAllHWBuffers is called in endScene, which means it is called at least once per frame, for RTTs even more often. This results in (hopefully...) 60 times per second, resulting in about 5.5 minutes until deletion. Something which should be configurable, but not too far from a sensable value.
One thing we should probably promote somewhat more intensive is the manual buffer management.
But at least we don't have any mem leak here, since the link is automatically removed during the app run, or at least at the end in the driver destructor.
Post Reply