1

I'm learning OpenGL by putting together various examples. My aim is to have an application that displays a 2D image, pixel-for-pixel (i.e. without geometric transformations) while allowing me to manipulate the pixel values from inside a fragment shader. I think I'm getting close to this goal, but my textures are getting spatially mangled and I can't see why.

My complete code is below. It works as I expect if I use the line marked OPTION A in my fragment shader: it gives me the ability to manipulate pixel values in image coordinates. The problem comes when I uncomment OPTION B, which tries to source fragment colour values via texture2D from an existing texture declared as uniform sampler2D.

The problem seems to be rooted in how I create the texture. My source for the image is a floating-point numpy.array with 3 layers (RGB). I'm trying the simplest test pattern I can think of: a black texture with the first half of the pixels turned red (see the line commented with TEST). The pixels come out red, so I'm apparently correctly addressing layers. But "the first half" of the pixels do not render as a coherent block of rows or columns (either of which I might have expected) but rather in the following very bizarre pattern: (a very bizarre pattern)

Quite apart from my main problem with pixel layout, code-review style comments are also very welcome because I'm sure there are things that I don't need to and/or shouldn't be doing. Please forgive the legacy GLSL code/OpenGL approach which seems to be necessary to support my 2013 MacBook running El Capitan (GLSL v. 1.20) in addition to my more modern-OpenGL capable machines that run Windows 10 (GLSL v. 4.x).

# Display framework adapted from:
#     - Pygame/PyopenGL examples by Bastiaan Zapf, Apr 2009 ( http://python-opengl-examples.blogspot.com/ )
# 
# Shader code for passing texture coordinates from vertex to fragment shader and reading from a uniform sampler2D, adapted from:
#     - "Modern OpenGL Textures" tutorial by Tom Dalling at http://www.tomdalling.com/blog/modern-opengl/02-textures/ (but note that we're really trying to support legacy OpenGL, not "modern")
# 
# (Failing?) attempt to make texture data accessible to the shader in the first place, adapted from:
#     - SO question by GergelyH at http://stackoverflow.com/questions/43482690


import sys
import time
import random
from math import * # trigonometry

import numpy

import pygame, pygame.locals # just to get a display
import OpenGL
from OpenGL.GL import *
from OpenGL.GLU import *

canvasWidth, canvasHeight = 800,600

# get an OpenGL surface
pygame.init() 
pygame.display.set_mode( ( canvasWidth, canvasHeight ), pygame.OPENGL | pygame.DOUBLEBUF )

glEnable(GL_DEPTH_TEST)
GLSL_Version = glGetString(GL_SHADING_LANGUAGE_VERSION)
print( 'GLSL Version = ' + GLSL_Version )
print( 'PyOpenGL Version = ' + OpenGL.__version__ )

def Shader(type, source):
    shader = glCreateShader(type)
    glShaderSource(shader, source)
    glCompileShader(shader)
    result = glGetShaderiv(shader, GL_COMPILE_STATUS)
    if result != 1: raise Exception( "Shader compilation failed:\n" + glGetShaderInfoLog(shader))
    return shader

vertex_shader = Shader(GL_VERTEX_SHADER, """\
varying vec2 fragTextureCoordinate;
void main(void)
{
    fragTextureCoordinate = gl_Vertex.xy;
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex; // TODO: is there a way of shortcutting this matrix multiplication...?
}
""");

fragment_shader = Shader(GL_FRAGMENT_SHADER, """\
uniform sampler2D tex;
varying vec2 fragTextureCoordinate;
void main(void)
{
    gl_FragColor = vec4(
        fragTextureCoordinate.x > %f, // right half red/yellow
        fragTextureCoordinate.y > %f, // top half green/yellow
        abs( fragTextureCoordinate.x - fragTextureCoordinate.y ) < 0.5, // diagonal blue/cyan/white line
        1  // opaque
    ); // OPTION A:  pixel-addressing sanity check.  This behaves as expected.  I can address the image in image coordinates, although note that (0,0) is bottom left

    gl_FragColor = texture2D( tex, fragTextureCoordinate );
    // OPTION B: texture mapping attempt.  The channels seem to be arranged sanely, but the spatial ordering of the pixels is really strange
}
""" % ( canvasWidth / 2.0, canvasHeight / 2.0 ) );
# build shader program
program = glCreateProgram()
glAttachShader( program, vertex_shader )
glAttachShader( program, fragment_shader )
glLinkProgram( program )
# try to activate/enable shader program, handling errors wisely
try:
    glUseProgram(program)   
except OpenGL.error.GLError:
    print( glGetProgramInfoLog( program ) )
    raise


gluOrtho2D(0, canvasWidth, 0, canvasHeight)
glClearColor(0.0, 0.0, 0.0, 1.0)

import numpy;
#img = numpy.zeros( [ canvasHeight, canvasWidth, 3 ], 'float32' ); ; img[ :, canvasWidth//2:,  0 ] = 0; img[ canvasHeight//2:, :, 1 ] = 0; img[ :, :, 2 ] = 0

img = numpy.zeros( [ canvasHeight * canvasWidth * 3 ], 'float32' )  # dtype='float32' here corresponds to type=GL_FLOAT and internalFormat=GL_RGB32F below
img[ : img.size//2 : 3 ] = 1.0   # TEST: light up every third pixel (i.e. red channel only) up until halfway through the image

######
# texture-loading section adapted from question by GergelyH at http://stackoverflow.com/questions/43482690
glActiveTexture( GL_TEXTURE0 )
textureID = glGenTextures( 1 )
glBindTexture( GL_TEXTURE_2D, textureID )
glEnable( GL_TEXTURE_2D )
glPixelStorei( GL_UNPACK_ALIGNMENT, 1 ) # no apparent effect
glTexImage2D( GL_TEXTURE_2D, 0, GL_RGB32F, canvasWidth, canvasHeight, 0, GL_RGB, GL_FLOAT, img.tostring() )
glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST )
glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST )
glTexEnvf( GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_DECAL )   # no apparent effect
try: glGenerateMipmap( GL_TEXTURE_2D )   # NB: seems to be available in PyOpenGL 3.0.2 but not 3.0.1; has no apparent effect
except NameError, err: print( err )
######

glUniform1i( glGetUniformLocation( program, "tex" ), 0 ) # set to 0 because the texture will be bound to GL_TEXTURE0, says Tom Dalling (that means it looks like we could only ever do this with 32 textures max)

# create display list
glNewList( 1, GL_COMPILE )
glBegin( GL_QUADS )
glColor3f( 1, 1, 1 )

glNormal3f(  0,            0,            1 )
glVertex3f(  0,            canvasHeight, 0 )
glVertex3f(  canvasWidth,  canvasHeight, 0 )
glVertex3f(  canvasWidth,  0, 0 )
glVertex3f(  0,            0, 0 )
# NB: unlike GergelyH's answer to his own question, it seems to make no difference
#     whether we call glTexCoord here---presumably because we're trying to do all the
#     texture mapping in the shader itself

glEnd()
glEndList()

done = False
pygame.event.get() # flush the event queue
while not done:
    glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT )
    glCallList( 1 )
    #print( Time() - t )
    pygame.display.flip()
    time.sleep( 0.014 ) # TODO: hardcoded

    # Press 'q' to quit or 's' to save a timestamped snapshot
    for event in pygame.event.get():
        if event.type  == pygame.locals.QUIT: done = True
        elif event.type == pygame.locals.KEYUP and event.key in [ ord( 'q' ), 27 ]: done = True
        elif event.type == pygame.locals.KEYUP and event.key in [ ord( 's' ) ]:
            try: from PIL import Image
            except ImportError: import Image
            filename = time.strftime('snapshot-%Y%m%d-%H%M%S.png')
            rawRGB = glReadPixels( 0, 0, canvasWidth, canvasHeight, GL_RGB, GL_UNSIGNED_BYTE )
            Image.frombuffer( 'RGB', [ canvasWidth, canvasHeight ], rawRGB, 'raw', 'RGB', 0, 1 ).transpose( Image.FLIP_TOP_BOTTOM ).save( filename )

pygame.display.quit()
jez
  • 14,867
  • 5
  • 37
  • 64
  • All [2013 MacBooks](https://support.apple.com/en-us/HT202823) should do OpenGL 4.1 (Core context, naturally) just fine. – genpfault Jun 14 '17 at 01:04
  • @genpfault Perhaps it's the fact that I haven't (and am unlikely to be able to any time soon) upgraded the OS beyond El Capitan. Or perhaps it's something else—either way, it reports GLSL version 1.20 and the compiler does not like `in` and `out` variables. – jez Jun 14 '17 at 01:13
  • I have no clue about python but you're passing `img.tostring()` to glTexImage2D ... what does `tostring` return ?! As to your subquestion regarding cutting matrix multiplication, yes, just define your vertices in NDC(-1,+1) range, that being said, since you're using oldschool GL here you could just drop the vertexshader and vertex list stuff and just use `glRects` to make your draw. – LJᛃ Jun 15 '17 at 00:58

0 Answers0