Failed to compile pixel shader (linux - works fine in windows) [SOLVED]

[raidzero]

Hello, I just built Dolphin 3.5-397 from git, since my package manager does not have dolphin in it. I installed gcc 4.6.2 alongside my standard 4.5.3 in order to build.

This is on Gentoo Linux x64 kernel version 3.5.4.

core i7 930 @ 4ghz
24gb ddr3-1600 g-skill ram
radeon hd 5850 1gb (which runs dolphin fine in windows)

When I try to launch any game, I tried Tetris Worlds and Metroid Prime (Gamecube) also Metroid Prime Trilogy (Wii), I see a "failed to compile pixel shader" error.

Apparently using code tags smashes newlines, so I used pastebin to paste my error message and file contents: http://pastebin.com/UXNS0WKs

EDIT: added console output: http://pastebin.com/HSFVeQ6e

Any ideas? THANKS

[Shonumi]

Some games don't like having "Per-Pixel Lighting" enabled in the latest revisions. Check to make sure that it's off when playing for those games you mentioned.

If you still get issues, try to provide your settings for both Linux and Windows (as screenshots preferably).

[raidzero]

Shonumi, I am at work today so I cannot get screenshots, but I did ssh to my system and checked the ini files for both Windows and Linux, I found this parameter:

EnablePixelLighting.

Linux:
gfx_opengl.ini: EnablePixelLighting = False

Windows:
gfx_dx11.ini: EnablePixelLighting = False
gfx_dx9.ini: EnablePixelLighting = False

If this is not the corresponding parameter to "Per-Pixel Lighting", sorry. I will post screenshots when I can.

By the way, I probably should have included this in the OP; I am running fglrx 12.6.

[Shonumi]

Do the latest versions of fglrx fare any better? What about open-source Radeon drivers? Most of the time, your drivers should not be the issue, but we should at least eliminate them as possible problems. Though you're about the third Gentoo user I've seen running into graphical troubles with an AMD GPU.

Has Dolphin ever worked for you before? If so, what was the last revision you tried?

[neobrain]

Something is clearly going wrong here on our side. Someone should probably check where the instruction limit is being set to zero because, actually, we're forcing it to 4096 when it's reported to be zero by the driver.

@ OP: In Plugins/Plugin_VideoOGL/Src/PixelShaderCache.cpp, exchange the line http://code.google.com/p/dolphin-emu/source/browse/Source/Plugins/Plugin_VideoOGL/Src/PixelShaderCache.cpp#255 with "sprintf(stropt, "MaxLocalParams=224");"

[raidzero]

Shonumi, Yes dolphin 3.0 (not sure what revision) worked on my system, though it was really really, really bad. I'll update fglrx if it comes down to it.

neobrain, thanks for the stropt suggestion, I applied and rebuilt. Will have to see if it works when I get home, can't run 3d accelerated things over ssh X forwarding

[raidzero]

I tried neobrain;s suggestion but it didn't make a difference. Error message and file contents: http://pastebin.com/MLgQ9qUd

[neobrain]

Try replacing with sprintf(stropt, "MaxLocalParams=224,NumInstructionSlots=4096"); then?

[raidzero]

(02-14-2013, 03:21 AM)neobrain Wrote: Try replacing with sprintf(stropt, "MaxLocalParams=224,NumInstructionSlots=4096"); then?

, no luck. I also receive a failed to compile vertex shader message that I have been forgetting to mention, here are both error messages and both dump txt files:
http://pastebin.com/xqQ9iC32

EDIT: I adjusted the stropt sprintf line in VertexShaderCache.cpp accordingly but it made no difference:

Source/Plugins/Plugin_VideoOGL/Src/VertexShaderCache.cpp: sprintf(stropt, "MaxLocalParams=256,NumInstructionSlots=4096");
Source/Plugins/Plugin_VideoOGL/Src/PixelShaderCache.cpp: sprintf(stropt, "MaxLocalParams=224,NumInstructionSlots=4096");

EDIT 2: I had some doubt that it was actually replacing the dolphin-emu executable with the modified version, so I made a minor change (changed 4096 to 4095 in the pixel shader stropt line) and rebuilt and the checksum of dolphin-emu has indeed changed.

---

Found this
http://forums.dolphin-emu.org/archive/index.php?thread-8606.html

and it solved my issue, games now load

workaround is to just not call cgGLSetOptimalOptions() in Render.cpp for ATI

[raidzero]

I'm not a big fan of just commenting stuff out like that, it worked in my case but it would break others... I have CG version 2.1 on my system, and in Render.cpp there is #if that checks if CG_VERSION_NUM is 2100 (which it should be, for me) but CG_VERSION_NUM is #defined to be 3000 in cg.h. Shouldn't cmake have caught that I have an insufficient version of nvidia CG toolkit if its just expected to be 3.0? I dont know if cmake checks stuff like a typical configure script or just blindly creates makefiles