I am attempting to write a Flutter plugin for Android to allow me to directly write pixels using a Texture, so I need to make a SurfaceTexture available, and I want to be able to draw arbitrary pixel data to it using a single textured quad. For now, for debugging, I am simply trying to draw a single cyan triangle over a magenta background to verify my vertices are being drawn correctly, but it appears they are not. The glClear call is doing what I expect, as the magenta background is being shown instead of the black color that would otherwise be behind it, and I can change that color by changing what I pass to glClearColor, so in some way, the texture is being rendered, but I see no evidence that calling glDrawArrays is accomplishing anything. The code containing all of my interfacing with OpenGL ES is in the file below, and the drawTextureToCurrentSurface method is where both glClear and glDrawArrays are being called:
class EglContext {
companion object {
// Pass through position and UV values
val vertexSource = """
#version 300 es
precision mediump float;
/*layout(location = 0)*/ in vec2 position;
/*layout(location = 1)*/ in vec2 uv;
out vec2 uvOut;
void main() {
gl_Position = vec4(position, -0.5, 1.0);
uvOut = uv;
}
""".trimIndent()
// Eventually get the texture value, for now, just make it cyan so I can see it
val fragmentSource = """
#version 300 es
precision mediump float;
in vec2 uvOut;
out vec4 fragColor;
uniform sampler2D tex;
void main() {
vec4 texel = texture(tex, uvOut);
// Effectively ignore the texel without optimizing it out
fragColor = texel * 0.0001 + vec4(0.0, 1.0, 1.0, 1.0);
}
""".trimIndent()
var glThread: HandlerThread? = null
var glHandler: Handler? = null
}
private var display = EGL14.EGL_NO_DISPLAY
private var context = EGL14.EGL_NO_CONTEXT
private var config: EGLConfig? = null
private var vertexBuffer: FloatBuffer
private var uvBuffer: FloatBuffer
//private var indexBuffer: IntBuffer
private var defaultProgram: Int = -1
private var uniformTextureLocation: Int = -1
private var vertexLocation: Int = -1
private var uvLocation: Int = -1
var initialized = false
private fun checkGlError(msg: String) {
val errCodeEgl = EGL14.eglGetError()
val errCodeGl = GLES30.glGetError()
if (errCodeEgl != EGL14.EGL_SUCCESS || errCodeGl != GLES30.GL_NO_ERROR) {
throw RuntimeException(
"$msg - $errCodeEgl(${GLU.gluErrorString(errCodeEgl)}) : $errCodeGl(${
GLU.gluErrorString(
errCodeGl
)
})"
)
}
}
init {
// Flat square
// Am I allocating and writing to these correctly?
val vertices = floatArrayOf(-1f, -1f, 1f, -1f, -1f, 1f, 1f, 1f)
vertexBuffer = ByteBuffer.allocateDirect(vertices.size * 4).asFloatBuffer().also {
it.put(vertices)
it.position(0)
}
val uv = floatArrayOf(0f, 0f, 1f, 0f, 0f, 1f, 1f, 1f)
uvBuffer = ByteBuffer.allocateDirect(uv.size * 4).asFloatBuffer().also {
it.put(uv)
it.position(0)
}
// Not being used until I can figure out what's currently not working
/*val indices = intArrayOf(0, 1, 2, 2, 1, 3)
indexBuffer = ByteBuffer.allocateDirect(indices.size * 4).asIntBuffer().also {
it.position(0)
it.put(indices)
it.position(0)
}*/
if (glThread == null) {
glThread = HandlerThread("flutterSoftwareRendererPlugin")
glThread!!.start()
glHandler = Handler(glThread!!.looper)
}
}
// Run OpenGL code on a separate thread to keep the context available
private fun doOnGlThread(blocking: Boolean = true, task: () -> Unit) {
val semaphore: Semaphore? = if (blocking) Semaphore(0) else null
glHandler!!.post {
task.invoke()
semaphore?.release()
}
semaphore?.acquire()
}
fun setup() {
doOnGlThread {
Log.d("Native", "Setting up EglContext")
display = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
if (display == EGL14.EGL_NO_DISPLAY) {
Log.e("Native", "No display")
checkGlError("Failed to get display")
}
val versionBuffer = IntArray(2)
if (!EGL14.eglInitialize(display, versionBuffer, 0, versionBuffer, 1)) {
Log.e("Native", "Did not init")
checkGlError("Failed to initialize")
}
val configs = arrayOfNulls<EGLConfig>(1)
val configNumBuffer = IntArray(1)
var attrBuffer = intArrayOf(
EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT,
EGL14.EGL_RED_SIZE, 8,
EGL14.EGL_GREEN_SIZE, 8,
EGL14.EGL_BLUE_SIZE, 8,
EGL14.EGL_ALPHA_SIZE, 8,
EGL14.EGL_DEPTH_SIZE, 16,
//EGL14.EGL_STENCIL_SIZE, 8,
//EGL14.EGL_SAMPLE_BUFFERS, 1,
//EGL14.EGL_SAMPLES, 4,
EGL14.EGL_NONE
)
if (!EGL14.eglChooseConfig(
display,
attrBuffer,
0,
configs,
0,
configs.size,
configNumBuffer,
0
)
) {
Log.e("Native", "No config")
checkGlError("Failed to choose a config")
}
if (configNumBuffer[0] == 0) {
Log.e("Native", "No config")
checkGlError("Got zero configs")
}
Log.d("Native", "Got Config x${configNumBuffer[0]}: ${configs[0]}")
config = configs[0]
attrBuffer = intArrayOf(
EGL14.EGL_CONTEXT_CLIENT_VERSION, 2, EGL14.EGL_NONE
)
context = EGL14.eglCreateContext(display, config, EGL14.EGL_NO_CONTEXT, attrBuffer, 0)
if (context == EGL14.EGL_NO_CONTEXT) {
Log.e("Native", "Failed to get any context")
checkGlError("Failed to get context")
}
Log.d("Native", "Context = $context\n 'Current' = ${EGL14.eglGetCurrentContext()}")
initialized = true
}
}
// Called by my plugin to get a surface to register for Texture widget
fun buildSurfaceTextureWindow(surfaceTexture: SurfaceTexture): EGLSurface {
var _surface: EGLSurface? = null
doOnGlThread {
val attribBuffer = intArrayOf(EGL14.EGL_NONE)
val surface =
EGL14.eglCreateWindowSurface(display, config, surfaceTexture, attribBuffer, 0)
if (surface == EGL14.EGL_NO_SURFACE) {
checkGlError("Obtained no surface")
}
EGL14.eglMakeCurrent(display, surface, surface, context)
Log.d("Native", "New current context = ${EGL14.eglGetCurrentContext()}")
if (defaultProgram == -1) {
defaultProgram = makeProgram(
mapOf(
GLES30.GL_VERTEX_SHADER to vertexSource,
GLES30.GL_FRAGMENT_SHADER to fragmentSource
)
)
uniformTextureLocation = GLES30.glGetUniformLocation(defaultProgram, "tex")
vertexLocation = GLES30.glGetAttribLocation(defaultProgram, "position")
uvLocation = GLES30.glGetAttribLocation(defaultProgram, "uv")
Log.d("Native", "Attrib locations $vertexLocation, $uvLocation")
checkGlError("Getting uniform")
}
_surface = surface
}
return _surface!!
}
fun makeCurrent(eglSurface: EGLSurface, width: Int, height: Int) {
doOnGlThread {
GLES30.glViewport(0, 0, width, height)
if (!EGL14.eglMakeCurrent(display, eglSurface, eglSurface, context)) {
checkGlError("Failed to make surface current")
}
}
}
fun makeTexture(width: Int, height: Int): Int {
var _texture: Int? = null
doOnGlThread {
val intArr = IntArray(1)
GLES30.glGenTextures(1, intArr, 0)
checkGlError("Generate texture")
Log.d("Native", "${EGL14.eglGetCurrentContext()} ?= ${EGL14.EGL_NO_CONTEXT}")
val texture = intArr[0]
Log.d("Native", "Texture = $texture")
GLES30.glBindTexture(GLES30.GL_TEXTURE_2D, texture)
checkGlError("Bind texture")
val buffer = ByteBuffer.allocateDirect(width * height * 4)
GLES30.glTexImage2D(
GLES30.GL_TEXTURE_2D,
0,
GLES30.GL_RGBA,
width,
height,
0,
GLES30.GL_RGBA,
GLES30.GL_UNSIGNED_BYTE,
buffer
)
checkGlError("Create texture buffer")
_texture = texture
}
return _texture!!
}
private fun compileShader(source: String, shaderType: Int): Int {
val currentContext = EGL14.eglGetCurrentContext()
val noContext = EGL14.EGL_NO_CONTEXT
val shaderId = GLES30.glCreateShader(shaderType)
Log.d("Native", "Created $shaderId\nContext $currentContext vs $noContext")
checkGlError("Create shader")
if (shaderId == 0) {
Log.e("Native", "Could not create shader for some reason")
checkGlError("Could not create shader")
}
GLES30.glShaderSource(shaderId, source)
checkGlError("Setting shader source")
GLES30.glCompileShader(shaderId)
val statusBuffer = IntArray(1)
GLES30.glGetShaderiv(shaderId, GLES30.GL_COMPILE_STATUS, statusBuffer, 0)
val shaderLog = GLES30.glGetShaderInfoLog(shaderId)
Log.d("Native", "Compiling shader #$shaderId : $shaderLog")
if (statusBuffer[0] == 0) {
GLES30.glDeleteShader(shaderId)
checkGlError("Failed to compile shader $shaderId")
}
return shaderId
}
private fun makeProgram(sources: Map<Int, String>): Int {
val currentContext = EGL14.eglGetCurrentContext()
val noContext = EGL14.EGL_NO_CONTEXT
val program = GLES30.glCreateProgram()
Log.d("Native", "Created $program\nContext $currentContext vs $noContext")
checkGlError("Create program")
sources.forEach {
val shader = compileShader(it.value, it.key)
GLES30.glAttachShader(program, shader)
}
val linkBuffer = IntArray(1)
GLES30.glLinkProgram(program)
GLES30.glGetProgramiv(program, GLES30.GL_LINK_STATUS, linkBuffer, 0)
if (linkBuffer[0] == 0) {
GLES30.glDeleteProgram(program)
checkGlError("Failed to link program $program")
}
return program
}
// Called to actually draw to the surface. When fully implemented it should draw whatever is
// on the associated texture, but for now, to debug, I just want to verify I can draw vertices,
// but it seems I cannot?
fun drawTextureToCurrentSurface(texture: Int, surface: EGLSurface) {
doOnGlThread {
// Verify I have a context
val currentContext = EGL14.eglGetCurrentContext()
val noContext = EGL14.EGL_NO_CONTEXT
Log.d("Native", "Drawing, Context = $currentContext vs $noContext")
checkGlError("Just checking first")
GLES30.glClearColor(1f, 0f, 1f, 1f)
GLES30.glClearDepthf(1f)
GLES30.glDisable(GLES30.GL_DEPTH_TEST)
GLES30.glClear(GLES30.GL_COLOR_BUFFER_BIT or GLES30.GL_DEPTH_BUFFER_BIT)
checkGlError("Clearing")
GLES30.glUseProgram(defaultProgram)
checkGlError("Use program")
GLES30.glActiveTexture(GLES30.GL_TEXTURE0)
checkGlError("Activate texture 0")
GLES30.glBindTexture(GLES30.GL_TEXTURE_2D, texture)
checkGlError("Bind texture $texture")
GLES30.glUniform1i(uniformTextureLocation, 0)
checkGlError("Set uniform")
GLES30.glEnableVertexAttribArray(vertexLocation)
vertexBuffer.position(0)
GLES30.glVertexAttribPointer(vertexLocation, 2, GLES30.GL_FLOAT, false, 0, vertexBuffer)
Log.d("Native", "Bound vertices (shader=$defaultProgram)")
checkGlError("Attribute 0")
GLES30.glEnableVertexAttribArray(uvLocation)
uvBuffer.position(0)
GLES30.glVertexAttribPointer(uvLocation, 2, GLES30.GL_FLOAT, false, 0, uvBuffer)
checkGlError("Attribute 1")
//indexBuffer.position(0)
//GLES30.glDrawElements(GLES30.GL_TRIANGLES, 4, GLES30.GL_UNSIGNED_INT, indexBuffer)
// I would expect to get a triangle of different color than the background
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 3)
GLES30.glFinish()
checkGlError("Finished GL")
EGL14.eglSwapBuffers(display, surface)
checkGlError("Swapped buffers")
}
}
...currently unused other methods
}
The general flow of the above code is that the init block executes when initializing the context, of which there is only one. setup is called when the plugin is registered, and buildSurfaceTextureWindow is called when initializing a SurfaceTexture for a Flutter Texture. The first time this is called, it compiles the shaders. When the plugin wants to render the texture, it calls makeCurrent then drawTextureToCurrentSurface, which is where the magenta background becomes visible but without any cyan triangle. Calls to GL functions are done in a separate thread using doOnGlThread.
If you need to see all of the code including the full plugin implementation and example app using it, I have it on Github, but as far as I can tell the above code should be the only relevant region to not seeing any geometry rendered in the effectively hardcoded color from my fragment shader.
tl;dr My background color from glClear shows up on screen, but my expected result of calling glDrawArrays, a cyan triangle, does not, and I am trying to understand why.
