note to self: windows is not optimal for benchmarking

Discussion about everything. New games, 3d math, development tips...
Post Reply
Cube_
Posts: 1010
Joined: Mon Oct 24, 2011 10:03 pm
Location: 0x45 61 72 74 68 2c 20 69 6e 20 74 68 65 20 73 6f 6c 20 73 79 73 74 65 6d

note to self: windows is not optimal for benchmarking

Post by Cube_ »

this is really rather off topic, but it's sorta annoying

So I built my game and tested it, 164FPS (some pseudo median/average of the actual numbers ranging from 140-178), fair enough - then I go and have a nice meal, I return to my computer being magically rebooted (seemingly a system crash based on firefox acting like it had shut down unexpectedly) - well, whatever. So I fire it up again and now the performance is magically 64 fps (avg, actual numbers 62-68 predominantly 64) - what's worse is that the system resource monitor just returns more or less nominal values, low memory usage, low pcu usage, low network usage - all core temperatures are fine, be it cpu or gpu - voltage levels are nominal, memory speed is nominal, and the gpu isn't being throttled as far as I can tell based on the driver and cpu-z.

The only environmental difference I haven't tested yet is that I don't have a youtube video playing currently, which I did during the initial testing but that really shouldn't cause higher framerates, unless the gpu throttles due to low load and decides that 64 fps is more than good enough and thus there's no need to go into the higher performance profile.

Ah well, there's little I can do about that aside from reinstall but I installed windows on this computer less than a month ago, that's just silly.
well, whatever I suppose this mini-rant is coming to an end, the only reason I'm posting it is because someone's bound to at least be able to verify one of my theories or provide some other useful metric I can try - in any case the build hasn't changed so I can exclude that as a potential difference.

EDIT: after some more digging I found that AMD Catalyst, in its infinite wisdom, decided to update to a known broken driver build (15.7.1) from my known good (15.6) build.

the problem with this is twofold:
This driver causes BSoDs with THREAD_STUCK_IN_DEVICE_DRIVER
and it runs significantly worse (a drop of about 110-120fps in my project currently).

The solution is even more obnoxious, I could uninstall catalyst 15.7.1 (well, 15.20.1062.1004 is the driver version - 15.7.1 is the catalyst release version) and reinstall 15.6 but it'll autoupdate again, there's no way to disable this behavior in current builds so the solution would essentially boil down to: downgrade, await auto-update, downgrade, await auto-update, downgrade

So on a more correct note: the way windows handles software and drivers makes it subpar for benchmarking because it is impossible to enforce consistent behavior without a complete reinstall of everything between each test - combine this with AMD's apparent ineptitude at good software design and we reach the inevitable conclusion that plenty of professional programmers are really bad at their jobs.

hopefully this </rant> actually applies - the last one didn't.
"this is not the bottleneck you are looking for"
Granyte
Posts: 850
Joined: Tue Jan 25, 2011 11:07 pm
Contact:

Re: note to self: windows is not optimal for benchmarking

Post by Granyte »

why in the world are you still running 15.6 and what gpu do you have?

Also only windows 10 force driver update and event that can be disabeled either by hiding the update or by selecting to prevent download for new drivers.

And most likely you issue on 15.7.1 are due to a failed install Amd's drivers are decent but thier installer is crap and often fails without a warning leaving you with a half working driver
Cube_
Posts: 1010
Joined: Mon Oct 24, 2011 10:03 pm
Location: 0x45 61 72 74 68 2c 20 69 6e 20 74 68 65 20 73 6f 6c 20 73 79 73 74 65 6d

Re: note to self: windows is not optimal for benchmarking

Post by Cube_ »

I was running 15.6 because that's the version I happened to stumble upon first when browsing AMD's dysfunctional downloads section, in any case I can't be bothered to try and fix it - the downgrade would work but the option to disable auto updates in catalyst is long since gone (it WAS there in 11.x but that's clearly quite a while ago).

As for my GPU, (well APU I suppose) A4-5000 (with Radeon HD 8330 graphics)
"this is not the bottleneck you are looking for"
Post Reply