4

I'm writing shader for unity3d. The shader uses multiple render targets to render post processing effect.

However, I've run into interesting issue.

When Unity3d runs in direct3d mode, by default all standard shaders write data only into first color buffer (i.e. with index 0). I.e. if I attach 3 color buffers to camera, call Camera.Render color buffer with index 0 will contain rendered scene, and all the other buffers will remain untouched unless some shader specifically write in them. My shader utilizes that behavior (I use buffers with indexes 1 and 2 to accumulate data needed for post process effect).

However, in OpenGL mode standard unity3d shaders write in ALL color buffers at once. I.e. if I attach multiple render buffers to a camera, call Camera.Render all 3 buffers will contain copy of rendered scene.

That breaks my shader in OpenGL mode.

How can I fix that? I need to render the whole scene in one go, and only objects that have specific shader should modify additional color buffers.

I need to render scene in one go because using layer masks causes unity to recalculate projector shadows for ALL lights and I need shadows to be correct.

Advice?

SigTerm
  • 26,089
  • 6
  • 66
  • 115
  • Have you tried manually manipulating command buffers on the camera? http://docs.unity3d.com/ScriptReference/Camera.GetCommandBuffers.html – Max Yankov May 09 '15 at 21:38
  • @MaxYankov: No, and it would help if you were more specific. I see that I can insert additional commands there, but I'm not sure what am I supposed to do with the buffers. I don't exactly remember if D3D/OGL document default MRT behavior anywhere. Basically, on D3D if shader writes to first buffer only, secondary buffes are untouched. On OGL they receive copy of information from 1st buffer. That's with standard unity shader. – SigTerm May 09 '15 at 22:11
  • I wish I could be more specific, but I don't think that I have enough experience in the matter. However, this is where I would dig in if I had this problem. – Max Yankov May 09 '15 at 22:12
  • @MaxYankov: I mean, I don't know any gl command that would change default behavior for additional color buffers. – SigTerm May 09 '15 at 22:13
  • @MaxYankov: :-\ Alright. I'll check GL spec tomorrow, but I somehow doubt that I'll find anything useful. The problem is that unity does not let me precisely control what is rendered, when and how. So if I start using layers, it recalculates shadowmaps for ALL visible lights, which causes shadows to disappear. Replacement shader could work IF it allowed me to specify that some objects shohuld be drawn using their default shader. And, of course there is no source code available for `Camera.Render` in unity personal. Sigh. – SigTerm May 09 '15 at 22:16
  • Well, it is supposed to be an abstraction layer over that stuff, right? However, it's often leaky, especially in the rendering department. Please post the solution if you find it. – Max Yankov May 09 '15 at 22:21

1 Answers1

1

Sadly, it turned out that "not writing into one of the render targets" is undocumented behavior in opengl. Standard unity shader when compiled for forward rendering path produces gl_FragData[0] = ...; assignment and writes into only one buffer, which triggers undocumented behavior and causes the mess.

In order to fix that problem, I would need to make unity write data explicitly into additional render targets in standard shaders. Unfortunately, this cannot be done, because there is no "entry point" to "hook" standard shader and write additional data into other color buffers. The closest thing to that is "finalcolor" modifier, but it does not actually allow to write into additional buffers via CG shader (that requires additional data to be from fragment shader, which is inaccessible from surface shader), it is only possible to modify one color.

I decided to rewrite portion of the shader (so it won't trigger undocumented behavior in OpenGL) and gave up on having unity shadowmap support in the effect. As far as I know, there is no other options short of modifying unity engine (requires "special arrangements" and source code access) or replacing entire lighting system with my own.

SigTerm
  • 26,089
  • 6
  • 66
  • 115