0

Getting into opengl since a short while, I use a fragment shader to render images loaded as textures, with some pixel-wise transformations like brightness and contrasts. I am using openGL 4.1, on Mac Os X , with intel Iris graphics.
Now I need to be able to quickly see through large images (>40 Mpx images). I'd like to simply animate them as fast possible, in addition to stop within any of them and do some GPU work on them, avoiding any round trip to the CPU.

Currently, dealing with one image at a time, my fragment shader is as follows:

#version 410 core

in mediump vec3 ourColor;
in mediump vec2 TexCoord;

out vec4 color;

uniform mediump sampler2D ourTexture;
uniform mediump float alpha;
uniform mediump float beta;
uniform mediump float gamma;

void main()
{
    mediump vec4 textureColor = texture(ourTexture, TexCoord);
    mediump vec3 scaledRGB = alpha * textureColor.rgb + beta;
    mediump vec3 gammaVec = vec3(gamma);
    scaledRGB = pow(scaledRGB, gammaVec);
    color = vec4(scaledRGB, textureColor.a);
}

I have read so far (e.g. here and there) that one would use arrays of samplers, and using uniform index for telling which texture will actually be in use, with the limitation set by GL_MAX_TEXTURE_IMAGE_UNITS regarding the max number of textures that we can use "at once" in the GPU (unless I misunderstood?). Is there a way to bypass this texture limitation? i.e. any way to send as many 2D arrays of floats to the GPU as the memory allows it, regardless of the texture image units limit, that I can later use as textures with the shaders? The hardware i'm targeting have GL_MAX_TEXTURE_IMAGE_UNITS = 16 (on my Intel Iris) or 32 typically, which is ridiculously small for time series of images that can be up to several hundreds. I am expecting that with recent GPUs of up to 2 GB of RAM, there must be a way to store more images than GL_MAX_TEXTURE_IMAGE_UNITS.

Am I missing a very simple alternative?

Thanks

Community
  • 1
  • 1
Wall-E
  • 623
  • 5
  • 17

1 Answers1

2

The GL_MAX_TEXTURE_IMAGE_UNITS is the maximum of different texture units which can be in use at once during a single draw call. This means that the GPU (or more precisely, your fragment shader - there are separate limits for the other programmable stages) can sample from up to that many different textures and combine the results arbitrarily.

It is not a limit of how many textures the GPU can manage. You can typically create as many textures as memory allows - and that does usually not only inlclude VRAM, but also system memory and even swap memory/page file (although that could get ridiculously slow).

Since you need to draw only a single frame at once, you only need a single texture unit. You just bind a different texture to the texture unit each frame. It is unclear to me what you mean by "avoiding any round trip to the CPU". You can't animate the texture on the GPU alone. You will need at least a draw call per frame, and a call to swap buffers inbetween. So binding a different texture each time will be easy.

Note that you can also use 2D array textures. These allow you to store an huge amount of 2D images as several "slices" of a single texture object. So your shader can access all of those images just by sampling with the appropriate texture coords. Array textures require each slice to be the same size, but this should not pose any problems for your use case.

derhass
  • 43,833
  • 2
  • 57
  • 78
  • Thanks for these advise. I had forgotten about the 2D array textures and will have a close look, it may solve my problem. As for "avoiding any round trip to the CPU", I meant that with my current "single-image texture" fragment shader, my only way to go on with it was, for each new image to render, to resend new data stored in the RAM into the GPU memory (vram) with `glTexImage2D`. This would happen each time I need to send a new image into the gpu. It's not really a "round trip", it's just one way, from ram to vram, but that's the bottleneck I have to avoid for quickly animating large images. – Wall-E Nov 21 '15 at 23:59
  • So, from what you're advising, let's say I define an array of sampler with: `uniform sampler2D ourTexture[NImages]`, then how shall I send the series of images with gl commands? So far, for an image loaded as a float 2D array called "image" in the ram, I would use `glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, image)` But it is unclear what this shall become to send the series of `NImages` all at once. – Wall-E Nov 22 '15 at 00:06
  • @user31412L I din't advise for _arrays of samplers_. I did suggest _array texturers_ which is a completely different concept. For the non-array case, you can simply create and upload all your textures during pre-processing. For the drawing, you simply need a `glBindTexture` before the draw call. For the array texture case, you simply load all images into a single array texture object during pre-processing, and select the layer via the 3rd component of the texcoords. You could use a uniform for that. Then you'll need one uniform update per frame - both approaches are quite similiar. – derhass Nov 22 '15 at 00:22
  • @user31412: No matter how you turn this stone, there's no way to avoid sending image data from the CPU to the GPU if you want to display it on the screen. The GPU is what sends data to the display device, so that data transfer has to happen anyway. If your GPU happens to sit on a dedicated graphics board, connected to a peripheral bus that data has to be copied unconditionally. And if your GPU sits on the same package as the CPU and shares system memory (think Intel integrated HD graphics, AMD APU), the data transfer may boil down to a copy-on-write page transfer. But it has to happen. – datenwolf Nov 22 '15 at 11:08
  • @datenwolf I was unclear. Of course I need to send things from the CPU to the GPU, I don't want to avoid that mandatory step. What i was saying is that I want to send the image series all at once and then only sending an index to the shader so that the GPU knows which image to use as the current texture to display, and at any given time when dealing with one given image from the series, all the images are already in the GPU. – Wall-E Nov 22 '15 at 11:22
  • @derhass Regarding the non-array case, are you referring to the GL_TEXTURE_3D type as defined [here](https://www.opengl.org/wiki/Texture)? – Wall-E Nov 22 '15 at 12:05
  • 1
    No, i'm speaking for _n_ _different_ `GL_TEXTURE_2D` objects - one per frame. – derhass Nov 22 '15 at 14:12
  • @user31412: You can upload an "arbitrary" (well, you're limited by the amount of available memory and the number range of GLuint) texture images to OpenGL. The key function here is `glBindTexture`. – datenwolf Nov 22 '15 at 20:55
  • @derhass @datenwolf Ok, so using `GL_TEXTURE_3D` is not even another alternative? Non-array case: for sending _n_ different `GL_TEXTURE_2D`, shall I proceed first with this? `GLuint texture[n];` and `glGenTextures(n, texture);` Then for sending my images as n different textures, I loop over _i_ with `glBindTexture(GL_TEXTURE_2D, texture[i]);` and send whatever `image_i` in the series at index i, with `glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, image_i);` Then unbind and iterate again until i've transferred all? – Wall-E Nov 22 '15 at 22:45
  • 1
    @user31412: `GL_TEXTURE_3D` is just a clumsy alternative to `GL_TEXTURE_2D_ARRAY`. It provides much stricter limits on the dimensions, like 1024x1024x1024, without providing any benefit in your use case. 3D textures allow filtering in all 3 dimensions, and that is something which you don't need at all. – derhass Nov 22 '15 at 22:47
  • 1
    @user31412: That's exactly how it goes. Thinks like 2D arrays or texture atlases are there for if you need to mix a lot of different images when drawing the very same primitive as there's only a limited number of texture units that can be active at the same time. But if your goal is to just display a bunch of different triangles or quads, each with a different image on it, then ou need only one texture unit and switch images for each primitive. – datenwolf Nov 23 '15 at 00:19
  • @user31412: Also note that the current generations of GPUs support bindless texturing, i.e. you no longer have to bind a texture to use it for drawing. However using bindless textures takes a more effort which is only justified if you really run into a texture bind management bottleneck. – datenwolf Nov 23 '15 at 00:19
  • @datenwolf It's working! Thanks! Now I need to make sure I do not load too many textures. How can I check that programmatically? Is there a function compatible with opengl that I can use to query how much GPU memory is available for openGL? Note that, in fact, I do all this within Qt 5. – Wall-E Nov 27 '15 at 12:30
  • 1
    @user31412: There is no such function in OpenGL, because you're only limited in *system memory* (GPU RAM + system RAM + swap space) in how many textures you can load. The RAM on a graphics card is just a very big cache. When you load textures into OpenGL they normally don't go to the GPU at first. Only when you actually make use of a texture for rendering, the texture will be loaded into GPU RAM. If you don't use a texture eventually it will be removed from GPU RAM. On modern GPUs in fact only the subset of a texture that's actually used is being loaded. – datenwolf Nov 27 '15 at 13:05
  • 1
    @user31412: Or in other words: Don't worry about clogging up the GPU RAM with textures. The only thing that matters is, that your working set, i.e. the set of textures you use to draw a whole frame fit into GPU RAM so that the rendering is not slowed down by the image transfers (and if framerates don't matter you can draw with a as large texture set as you want to). – datenwolf Nov 27 '15 at 13:06