Irrlicht's OWN file formats?

[xirtamatrix]

@Radikalizm: thanks for the detailed explanation.

Since I have never used this attribute system of Irrlicht, I'd leave the decision to use it or not to You and others hwo know better than me.
Myself, just took a look at this: http://www.parashift.com/c++-faq-lite/serialization.html#faq-36.6
Seems the endianness should'nt be such a big problem afterall.

Surely you're right that the format must also support all platforms Irrlicht supports.

[hybrid]

Just two things: Don't use STL if you want this format to make it into the engine. And endianess will be a problem, at least if your concern is speed. Conversions make access very slow. And it's up to you to decide whether Intel machines or ARM machines should get the overhead. However, you probably have no other choice for binary formats. Some platform will have to do the job.

[Radikalizm]

You will face endianess problems with every binary format you use. Whether with or without serialization. After all, serialization is just a structured way to put structured data into a long list, storing the structure information in this list. And of course also providing a way to get the data back. Doing all this in a uniform way with a common API is serialization, otherwise it's just I/O. For Irrlicht based apps using the interface is simple. For the loaders and exportes of other apps you have to mimick this process. But that's true also for many other formats. But for this file format, performance will be a very high priority. Otherwise Collada or FBX would already do the job. So deserializing large arrays (of indices and vertices) will be a major task to optimize. Not sure if the current attribute system would help.
BTW: We support the ogre mesh format...

You can always overcome the endianess problem in one way or another, but it won't be pretty
Maybe letting the developer decide for which platforms they want to compile the binary formats would be the best solution here, this would always guarantee performance if done right, but would obviously cause unwanted memory and pre-processing overhead

We're talking about making this Irrlicht's native animation format (i.e. based on skinnedmesh), which all modern loaders will use. To be future-proof we'd need to review all current animation formats, catalog the features they support, and design an animation playback model that supports most of them. Some things will incompatible with Irrlicht's innards, so we need a roadmap of what would need to change in future (flexible vertex format, shader loaders, level of detail controls and so on), what we're not bothering to support (IK, any form of physics) and how to access the stuff that we don't support (hit-boxes, collision meshes and so on).

IMO the hard part is gathering the requirements (reading and absorbing a library's worth of documents), because if they're wrong then the design is wrong from the start. Everything else is just writing code, which is the fun part!

I absolutely agree with you. Before we do any actual implementation we need to work on doing some decent research, and we need to officially decide what we will be supporting. Cataloging the features of existing formats would definitely be a good start IMO, but I wouldn't know how to make up the balance of which features to support vs what to modify or add to the engine to support these features
I can imagine that we don't want to break the current API too much so this could be some serious headache-material

[xirtamatrix]

BTW: We support the ogre mesh format...

Yes, I'm aware of that and thanks to You I was able to use it in my own personal project because that was the ONLY format for which I could find a decent updated 3dsMax exporter. http://www.ogremax.com/
And I was happy to pay the $250 from my own pocket as it saved me tons of headache.

In fact, that is how the whole idea of developing Irrlicht's own file formats came to my head, and THEN we could also create similar tools, targeted to commerical projects.

[RdR]

Yes, I'm aware of that and thanks to You I was able to use it in my own personal project because that was the ONLY format for which I could find a decent updated 3dsMax exporter. http://www.ogremax.com/
And I was happy to pay the $250 from my own pocket as it saved me tons of headache.

Offtopic:
Had to same problem for my project to find a decent exporter for Maya or 3DS Max.
Should take a look at FragMotion, which can export to alot of formats.
3DS Max / Maya -> .fbx (2010) -> FragMotion -> (.b3d etc)

[xirtamatrix]

Offtopic:
Had to same problem for my project to find a decent exporter for Maya or 3DS Max.
Should take a look at FragMotion, which can export to alot of formats.
3DS Max / Maya -> .fbx (2010) -> FragMotion -> (.b3d etc)

Thanks for the link. Yes, "I" might have happliy used it if it had support for Ogre's .mesh format because thats the format I'm using and beacuse in the case of my personal project we are back to one-guy-does-all paradigm.
But the question is, would I have used it for the commerical project of my company? NO!
As interesting as it is, its still a hobyst tool, laks tons of required features. As for using it just as format converter, the point to understand is that having to work with multiple formats one for static meshes another for animated IS a no no to begin with. Irrlicht needs to have ONE core format it supports, which is suitbale for static as well as animated meshes, together with all the other required features of a format.

Also, when putting together a project pipeline, the goal is to use as few tools as possible. In my workplace, the artists use zBrush and 3dsMax, export straight to Ogre's formats and thats it, goes straight to the development team. No middle-level tools required. Furthermore, OgreMax is not just an exporter, it also embeds an Ogre window into 3dsMax, which gives the artists immense control over the art they create.

[xirtamatrix]

IMO the hard part is gathering the requirements (reading and absorbing a library's worth of documents), because if they're wrong then the design is wrong from the start. Everything else is just writing code, which is the fun part!

Indeed, that is my view too. That is why I'm not hastily jumping into defining the format. I mean defining the format itself is not really the hard part, one could define a simple format within a day. The hard part is to first figure out what are the requirements and features that this format needs to support. And that is what I'm trying to figure out at present and for which I'd hugely appreciate any input from everyone in the core team invloved with loaders, animation, materials etc.

Having said that, we'd still start simple, and then add as we go, but we need to have some understanding of the requirements before we start.

So, anyone and everyone out there, if there is ANYTHING you can contribute, please speak up NOW!

[Virion]

any progress on this?

[Etory]

First, sorry for my poor English.

My humble opinion:

It's actually important to have a support for a good 3D object animation format with Irrlicht.

To support a format specific to Irrlicht, I don't know if this's the best thing to do. Indeed, we must create and maintain exporters for 3D softwares.

For example, just look at the passage of Blender 2.4 to 2.5/2.6 to see that this's a real problem. The changes on Blender API made 2.4x exporters unusable in 2.5/2.6. If the person who was create the exporter in Blender 2.4 is gone then you could no possibility to export to the format Irrlicht with new Blender version. We must also consider the risk.

I see two solutions:

1. Use a format suitable for existing games to benefit from mass effect and therefore have a greater chance of having an exporter
2. Create an independent converter of 3D software that converts one or more existing popular formats to the format specific to Irrlicht.

For features:

I'm new to Irrlicht but actually I use a cal3D format but unfortunately that becomes obsolete. That's interesting in CAL3D is the separation of functions in different files: there are mesh files, skeleton, animation, ...

We can combine multiple mesh files to a skeleton file, which allows me to compose my characters (RPG) based on the equipment they wear.

As animations are in different files, it's easy to add or edit an animation by adding a new animation file.

This management module is very useful for games with a lot of combination (equipment on the characters, a lot animation, ...) as the RPG. This is the case of my project.

By cons, management monolithic (all in the same file) will be easier to use but must be reserved for decors or games where the appearance of the characters is fixed (FPS, RTS, ...).

In this case, it's important to be able to link several skeletons to be able to assemble objects (such as character and weapons).

It's also important to separate the object (mesh / animation / skeleton) of its texture. we could change the texture of this object easily by the game program.

[Radikalizm]

any progress on this?

We're gathering info and doing research right now

[aaammmsterdddam]

my oppinion is:

Why not 'just' implement the *.blend format?

It is MY personal favourite

[Etory]

For me, .blend format isn't appropriate for games. this is a kind of memory dump of Blender.

[Radikalizm]

Please read the whole thread for answers to the "Why not use format x or y?" questions

[3DModelerMan]

.blend is too large. I would suggest looking into OpenCTM: http://openctm.sourceforge.net/ if we aren't going to write a new file format.

[CuteAlien]

OpenCTM makes no sense when the idea is supporting better support for animation, materials etc. And .blend is a similar idea (effective way of serializing internal structures) - but for Blender not for Irrlicht obviously and our internal structure is very different.

The way to do this is to understand Irrlicht core-structures first. So whoever works on that will have to start there (and note that I think the best way of learning about that is working with the engine for a while, maybe writing a few patches, etc...).