0

I'm trying to compile a C# game project which utilizes HLSL shaders. These shaders are part of the .csproj project, and therefore Visual Studio is supposed to compile them along with the code. The shader properties should be correct - shaders are of the right type, the shaders use recent enough shader model (5.1) etc. Despite that, Visual Studio doesn't build all of the shaders, and instead outputs two unhelpful errors:

error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Invalid operand type for operand #2 of opcode #107 (counts are 1-based).
error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Can't continue validation - aborting.

My first guess was that the shaders must be faulty in some way. I opened the VS2017 command prompt and as expected, manually running fxc produced the same errors for certain pixel shaders. However, I then decided to check if there are other versions of fxc installed:

C:\koodia\OpenSAGE\src\OpenSage.DataViewer.Windows\Shaders>where fxc
C:\Program Files (x86)\Windows Kits\10\bin\x86\fxc.exe
C:\Program Files (x86)\Windows Kits\10\bin\10.0.15063.0\x86\fxc.exe

The latter one is from a more recent Windows SDK - fxc /? reports the version as Direct3D Shader Compiler 10.1, while the first one is Direct3D Shader Compiler 10.0.10011.16384. I verified with a Powershell script that all of the shaders compile succesfully with the newer compiler. Then I compiled the project from command line with msbuild, which confirmed that msbuild (and therefore Visual Studio) is invoking the older fxc:

FxCompile:
  C:\Program Files (x86)\Windows Kits\10\bin\x86\Fxc.exe /nologo /Emain /T ps_5_1 /Fo obj\Debug\Shaders\MeshPS.cso /Zi /enable_unbounded_descriptor_tables Shade
  rs\MeshPS.hlsl
error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Invalid operand type for operand #2 of opcode #107 (counts are 1-based). [C:\koodia\OpenSAGE\src
\OpenSage.DataViewer.Windows\OpenZH.DataViewer.Windows.csproj]
error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Can't continue validation - aborting. [C:\koodia\OpenSAGE\src\OpenSage.DataViewer.Windows\OpenZH
.DataViewer.Windows.csproj]

Finally, to the question itself: How do I make sure msbuild uses the shader compiler from the latest installed Windows SDK?

I have installed the latest Windows SDK, the latest Windows updates and I've upgraded to Visual Studio 15.3.5, with no effect. I couldn't find anything online directly related to this issue. Visual Studio doesn't seem to have any settings related to Windows SDK or DirectX versions.

paavohtl
  • 23
  • 1
  • 7
  • What is the file version of `bin\x86\fxc.exe`? Maybe you still have some older SDK installed (e.g. 10.0.10240) and it serves as a base one or your project targets older SDK? – user7860670 Oct 08 '17 at 07:38
  • @VTT fxc.exe is 10.0.10240.16384. The newer one is 10.0.15063.468. – paavohtl Oct 08 '17 at 11:20
  • @VTT I uninstalled both versions of the SDK, and reinstalled the latest one. Msbuild now uses an even older version of `fxc`, from the Windows 8 SDK (`C:\Program Files (x86)\Windows Kits\8.0\bin\x86\Fxc.exe`). – paavohtl Oct 08 '17 at 14:01
  • Did you retarget your project? Msbuild should figure out correct SDK paths from project settings. – user7860670 Oct 08 '17 at 14:03
  • As mentioned in the original question, this is a C# project. VS doesn't seem to support (re)targetting for them. This is the context menu for the project: https://i.imgur.com/4Wwnp70.png – paavohtl Oct 08 '17 at 14:05
  • I've managed to build it and the oldest SDK I have is 10.0.14393. – user7860670 Oct 08 '17 at 14:49

1 Answers1

0

I managed to temporarily work around this issue with a dumb trick. I replaced fxc.exe and d3dcompiler_47.dll in C:\Program Files (x86)\Windows Kits\8.1\bin\x86 with the ones from C:\Program Files (x86)\Windows Kits\10\bin\10.0.15063.0\x86 (and backed up the originals).

It's seldom a good idea to hand-modify Windows SDKs, but in my use case it's likely that it's not going to break anything, as FXC should be backwards compatible.

I'm still looking for a better solution, or at least a confirmation whether my Windows + VS installation is somehow messed up, or if this is an actual bug / missing feature in msbuild.

paavohtl
  • 23
  • 1
  • 7
  • thanks for sharing your solution here, you could mark it as the answer, so it could help other community members who get the same issues. – Leo Liu Oct 12 '17 at 05:01
  • @Leo-MSFT I could, but I don't feel like the solution is good enough to be an "official" answer. It's hacky, and might case issues with updates or forwards-incompatible shaders. However, If a better solution doesn't emerge in the say, next few weeks, I'll mark this as the answer. – paavohtl Oct 13 '17 at 09:31