Irrlicht unit tests

You discovered a bug in the engine, and you are sure that it is not a problem of your code? Just post it in here. Please read the bug posting guidelines first.
wing64
Competition winner
Posts: 242
Joined: Wed Jul 23, 2008 2:35 am
Location: Thailand
Contact:

Re: Irrlicht unit tests

Post by wing64 »

SVN: 4154
Log: http://pastebin.com/22fmttPQ
PC: i7-930, ATI Readeon HD4670 with Catalyst 12.3, Ram 6 Gb
OS: Win7 64bit
Nadro
Posts: 1648
Joined: Sun Feb 19, 2006 9:08 am
Location: Warsaw, Poland

Re: Irrlicht unit tests

Post by Nadro »

I fixed an issues with an aero effects, which cause an errors in D3D tests. Now my platform passed 60/65 tests (in debug mode 61/65 -> loadScene doesn't work properly on my config in release mode)

Here is my results:
http://pastebin.com/izMQDRRR
Library helping with network requests, tasks management, logger etc in desktop and mobile apps: https://github.com/GrupaPracuj/hermes
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: Irrlicht unit tests

Post by CuteAlien »

I tested again last week with latest changes, but forgot posting. On Windows VS32 there are 4 tests fixed since last run ('drawPixel', 'textureRenderStates', 'draw2DImage', 'twodmaterial') on Linux one more fails now ('drawRectOutline').

Code: Select all

 
SVN Revision                4162/VS32    4162/Linux32  4166/Linux64
'matrixOps'                                                    f (fixed now)
'fast_atof'                                                    f (fixed now)
'writeImageToFile'                                             f (fixed now)
'ioScene'                                                      f (fixed now)
'drawPixel'                     ok                f            f
'drawRectOutline'                                 f            f
'material'                      f                 f            f
'renderTargetTexture'           f                 f            f
'textureFeatures'               f                 f            f
'textureRenderStates'           ok                f            f
'transparentMaterials'          f                 f            f (sometimes)
'antiAliasing'                  f  (fixed)        f            f
'draw2DImage'                   ok                f            f
'lights'                        ok                f            f
'twodmaterial'                  ok                f            f
'viewPort'                      ok                f            f
'mrt'                           ok                f            f
'projectionMatrix'              ok                f            f
'lightMaps'                     ok                f            f
'triangleSelector'              ok                f            f
 
Linux 32-bit: http://ideone.com/kXduu
Windows VS32: http://ideone.com/a3dVC
For comparison this time a new Linux-64 system: http://ideone.com/CLIxQ

New Linux system data:
NVIDIA Driver: 295.40
Linux-x86_64 on Debian Squeeze
8 GB Memory
GeForce GTX 550 TI
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: Irrlicht unit tests

Post by CuteAlien »

Most tests (at least on Linux64, probably also on Linux32) are because the screenshot function switches the colors red and blue. So changing GL_BGR to GL_RGB for ECF_R8G8B8 in createScreenShot in COpenGLDriver fixes 13 tests (yay!). Also fixes screenshots. Not sure why it works like this on Windows, but works there either way, so fixing this doesn't seem to break anything as far as I could see).

Reason I didn't fix it yet is that there seem to be more cases where it looks to me like rgb and bgr might be switched (just from looking at names). But that would probably need own tests before I mess up things there. Can you take a look Hybrid - you know more about OpenGL driver. Once this is done we're down to just 3-4 tests failing, getting there :-)
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
Foaly
Posts: 142
Joined: Tue Apr 15, 2014 8:45 am
Location: Germany

Re: Irrlicht unit tests

Post by Foaly »

63 of 69 tests passed.
http://pastebin.com/caTBThNQ

svn revision 4896
Windows 7 64 bit, Irrlicht 64 bit with unicode filesystem and without D3D8
16 GB RAM, 8 Cores 3.11 GHz

failing tests:
projectionMatrix
draw2DImage
antiAliasing
transparentMaterials
renderTargetTexture
drawVertexPrimitive
kklouzal
Posts: 343
Joined: Sun Mar 28, 2010 8:14 pm
Location: USA - Arizona

Re: Irrlicht unit tests

Post by kklouzal »

Windows 8.1 64-Bit; IntelHD-4000 & ATI Radeon-8700M; VS2013 Express; Build Target: Release-Fast FPU @32 Bits. Latest Irrlicht Trunk SVN @time of post. ATI-Catalyst Version 14.4
35 tests of 69 passed.
https://gist.github.com/kklouzal/e336adf010bb3bd04575
Dream Big Or Go Home.
Help Me Help You.
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: Irrlicht unit tests

Post by CuteAlien »

Thanks, the different image sizes is interesting. I'll run it again when I'm back to Irrlicht next week.
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
AReichl
Posts: 268
Joined: Wed Jul 13, 2011 2:34 pm

Re: Irrlicht unit tests

Post by AReichl »

Found something in "testUtils.cpp":

#if defined(_MSC_VER)
should be
#if defined(_MSC_VER) || defined(_IRR_WINDOWS_API_)

and it should be placed BEHIND
#include "testUtils.h" // + Irrlicht.h with _IRR_WINDOWS_API_

And because
#define TESTING_ON_WINDOWS
is only used locally once, we could simplify everything to:

#include <assert.h>
#include <memory.h>
#include <stdarg.h>
#include <stdio.h>

#include "testUtils.h" // + Irrlicht.h with _IRR_WINDOWS_API_

#if defined(_MSC_VER) || defined(_IRR_WINDOWS_API_)
#define _CRT_SECURE_NO_WARNINGS 1
#include <windows.h> // For OutputDebugString()
#endif // _MSC_VER || _IRR_WINDOWS_API_
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: Irrlicht unit tests

Post by CuteAlien »

Thanks looks better that way I guess. Checked in.
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
Coolcoder360
Posts: 2
Joined: Wed Jan 10, 2018 6:03 am

Re: Irrlicht unit tests

Post by Coolcoder360 »

I dunno if this is still a thing to post test results here,
but I ran the tests and passed 67 out of 70 tests on Archlinux, with Intel HD 3000 on an i7, 6GB RAM, on svn version r5600.
It failed tests 51, 53, and 58.
the log is here: https://ideone.com/IHGBIV
Compared to the last test results I see, that's quite good, good job to the dev team.
AReichl
Posts: 268
Joined: Wed Jul 13, 2011 2:34 pm

Re: Irrlicht unit tests

Post by AReichl »

Last time i run the tests, i had a difference between DirectX and OpenGL in some of them.
Also one test was different when compiling with VisualStudio or MinGW ( all on Windows ).
I think i solved some differences by adding a compiler flag for MinGW, but i have to check again.
I could repeat them and see what has changed since.
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: Irrlicht unit tests

Post by CuteAlien »

@Coolcoder360: Hm, have to run again here. This is with Irrlicht trunk? I'm a little confused because test-numbers seem to have changed (51 always fails on OpenGL, but it was test 50 in Irrlicht 1.8). I forgot to run them anyway after latest fixed for some OpenGL materials, might be some have to be adapted after fix.

@ AReichl: Never hurts to post them here once in a while. Even if it's just to see if something changed.
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
Coolcoder360
Posts: 2
Joined: Wed Jan 10, 2018 6:03 am

Re: Irrlicht unit tests

Post by Coolcoder360 »

Yes, this was with the latest svn trunk.

And to clarify the numbers with names:
51 - drawVertexPrimitive
53 - renderTargetTexture
58 - antiAliasing
juozas
Posts: 10
Joined: Sun Mar 27, 2022 12:58 pm
Location: Lithuania
Contact:

Re: Irrlicht unit tests

Post by juozas »

Some tests failing on my Ubuntu 20.04 LTS with GeForce 210 legacy card and nouveou driver. CPU: Pentium(R) Dual-Core E6300 @ 2.80GHz
Failing tests:
Test 53 'renderTargetTexture'
Test 58 'antiAliasing'
Ful log: https://gist.github.com/juozaspo/0cd745 ... 71644acb17
Сделано в СССР
CuteAlien
Admin
Posts: 9628
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: Irrlicht unit tests

Post by CuteAlien »

Thanks! I should give Nouveou another chance at some point. Rather glad to see you get so much working with it by now (last time I tried Nouveou on my Ubuntu Laptop it froze before even getting to the login screen, but been a while).
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
Post Reply