24

I have started a new game project, and have decided to learn and use OpenGL for it (project is being simultaneously developed on Windows and Linux). At the same time, I am also very interested in Test Driven Development, and am trying to put my best effort into writing my unit tests to lead design before any actual code.

However, I think my lack of knowledge might be tripping me up, and I keep hitting a wall in trying to write my unit tests for the "rendering" parts of the codebase. I'm hoping someone can give me some insight on how to continue.

I know that I need to unit test my interactions with OpenGL, and not OpenGL itself. The only way I can see to do this is to abstract OpenGL away from the rest of my code to some extent, either by intercepting OpenGL function calls, or by creating a whole new class interface, allowing me to create a mock version of that class for the tests. (Even better would be to abstract them to a set of non-class functions in a separate namespace rather than virtual class abstractions, but I don't see how I could mock that.)

However, as I'm still learning OpenGL, I only have a passing idea of what that abstraction should look like. For example, shall I wrap each OpenGL call, or group them into higher-level functions based on tasks to be accomplished? Thin wrappers would do little more than call a particular OpenGL function, so I wouldn't need to test them beforehand, but I could end up with a large number of functions to wrap. Then again, if I go too far the other way and group multiple OpenGL calls together by task, I feel like I'll end up where I started, having a large body of code using OpenGL that itself needs to be tested before use.

Where is the middle ground? How to I learn to use OpenGL while at the same time doing proper unit testing beforehand?

Nairou
  • 3,585
  • 5
  • 29
  • 47

6 Answers6

18

Properly testing rendering is not worth the effort. However, you can still use TDD for everything else and while designing your application.

Here's a great article about TDD, games, and OpenGL.

amro
  • 484
  • 2
  • 6
5

You cannot automatically test rendering part. For that you'll need an sentient being with ability to see and recognize images. Computer doesn't qualify.

You can automatically test for succesfull resource creation - VBO, textures, shaders, display lists - , you can unit test shaders for compilation errors, test math library, you can detect OpenGL errors, but you cannot test rendering part. The best thing you can do is to make some kind of test routine that will render a picture (probably animated and interactive) and ask if it looks right. The test will not be 100% reliable - might work on one hardware, and not on other, human may miss an error, etc.

For example, shall I wrap each OpenGL call,

No, it isn't worth it.

or group them into higher-level functions based on tasks to be accomplished?

It makes sense to write a few calls/classes for creation of "texture", "mesh", "shader program", make some kind of automatic way for getting uniform location (if you use shaders), maybe a few classes for automatic release of OpenGL resources (i.e. the ones with glDelete*** or with functions like glIsTexture), but this is all. Normally, you shouldn't introduce extra abstractions/classes, unless there is a need for them - because it will be extra work with no gain.

SigTerm
  • 26,089
  • 6
  • 66
  • 115
3

Treat the OpenGL just like you would a Database. I would initially start of with a single interface. As time goes by and you add more methods, then you can start breaking the single interface into multiple interfaces.

As already mentioned you can not use a standard TDD library to test the rendering. But it is possible. Think of it sort of like testing the rendering of HTML in a web client. When writing my HTML I don't TDD Firefox or Internet Explorer.

Gutzofter
  • 2,003
  • 23
  • 26
2

A lot of people seem to think graphics are impervious to traditional TDD/unittesting. This isn't true.

The issues surrounding TDD are that the images won't always be exactly the same. Things like antialiasing settings (which can be overridden by driver settings). Differences between how different GPUs render the same image (polygon unwinding orders and such). Also as your game goes through development things like meshes and textures might get changed or tweaked.

Also people think you often need to do it and eyeball the result prior to seeing if its ok then add it as a passing test case (which will break due to the above issues).

You could wrap all the OpenGL calls with logging ones and ensure that things are called as expected, but that doesn't test the graphics are actually valid it just tests that your program is making the calls you predict that it should make. That's not very useful and it will be quite a bit of work. Also it's going to be hard to maintain if you do things like optimize out unnecessary draw calls.

What I would recommend doing is decoupling your rendering output from your windowing system. What I mean by that is 'Render to Texture' or use FrameBuffer Objects. This also helps with do things like viewports and UI/HUD overlays.

Make a 'Window' object. Something that opens an actual desktop Window using Qt, SDL, SFML, GLUT, or whatever. And pass in some kind of a Renderer to it. You can do things like allow multiple renderers and specify rectangle coordinates (you could render a different view in the corner of the screen). Or maybe make one master renderer but inherit a version that allows it to have subrenderers. The Window object can also handle the input, this provides a nice way to inject a fake/mock input object, you can test if the player moves forward as expected.

This modular approach is the kind of thing that falls out of TDD, it could also could let your renderer have its own renderers (maybe you want to do a skybox separately, or a picture on a security monitor in a game, reflections, shadowmaps and so on). You can also easily render at a different resolution to the native desktop (which is useful for setting a fixed resolution in TDD and could just be something like 32x32 for simple operations, but is also useful for devices like Android where the resolutions are fixed and the GPUs have different capabilities. Nexus 10 not beefy enough to render your game at its XDPI? Scale it down. Also helps with Linux/Xorg which doesn't always allow you to set all resolutions the GPU does although thats less of a problem now days).

Once you have your FBO, make some functions to query pixel colour. Then you can query all your basic operations. Do your meshes render correctly? Test it. Make a mesh with a single white triangle and query which positions you would expect to see white and which should remain black when drawing that triangle. A 1x1 triangle in a 1x1 window should cover %50 diagonally so test pixels near the edge and both sides of the middle, the borders of the window and the pointy bits. Next combine two triangles into a rectangle and it should fill the screen, now all the pixels should be white. Now try moving it away from the camera and check that the borders are black but the centre is white, with a 1.0x1.0 rectangle in a 1.0x1.0 window at 1.0 away you can apply some simple geometry to see approximately where the borders should end. Test with a different color. Make a more complex mesh with a multicolored cube and test rotation. Make a test normal map.

You might be able to test lights by rendering a sphere and and checking that the left side it brighter than the right and if you add another light on the other side they are about the same (just add all the pixels on the left and all the pixels on the right and compare within a margin of error). Increase the light brightness and see if the sum of the pixels goes up. You can test the lighting algorithms by working out the expected color at a specific point on a flat plane. You could see that spectral highlighting is brightest at the expected area. For your normal/bump mapping make a test texture.

Just avoid anything really specific, for example don't test exactly on an edge. They could be antialiased or just off slightly. If your doing something like testing the left/right brightness for lighting, don't expect the pixel count to be the same number on different systems or even both sides (in a symmetrical scene), use an error margin. Don't use real data, make sure you use basic 'test' textures and meshes that won't change.

David C. Bishop
  • 6,437
  • 3
  • 28
  • 22
2

I'd suggest make a small prototype or some other little side-project and play around with opengl. This will give you some familiarity with it. Once that's done building your application using tdd should be much easier. And you're spot on with the fact that you need to mock opengl and not test opengl.

obelix
  • 986
  • 8
  • 14
0

Take a look at What is the best way to debug OpenGL?; the tools mentioned in the answers to this question will make some form of testing possible, especially GLIntercept (an OpenGL function call interceptor) sounds like a very interesting option/starting point for further investigations.

Community
  • 1
  • 1
Greg S
  • 12,333
  • 2
  • 41
  • 48