OnFrameAvailable callback is not coming

Question

I am new to android app and I am trying Camera using SurfaceTexture. The call back for OnFrameAvailable() is not being called... Please suggest me a solution. The code is below.

What is missing in this? I am not sure if I have made the correct call to setOnFrameListener().

package com.example.cameratest;

import com.example.test.R;

import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.view.Menu;
import android.view.View;


import android.graphics.SurfaceTexture;
import android.graphics.SurfaceTexture.OnFrameAvailableListener;
import android.hardware.Camera;
import android.media.MediaCodec;
import android.media.MediaCodecInfo;
import android.media.MediaFormat;
import android.media.MediaMuxer;
import android.opengl.*;

import android.util.Log;
import android.view.Surface;


import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import java.util.concurrent.locks.ReentrantLock;

public class MainActivity extends Activity implements OnFrameAvailableListener {
    private static final String TAG = "CameraToMpegTest";
    private static final boolean VERBOSE = true;           // lots of logging
    // where to put the output file (note: /sdcard requires WRITE_EXTERNAL_STORAGE permission)
    private static final long DURATION_SEC = 8;
    // camera state
    private Camera mCamera;
    private static SurfaceTexture mSurfaceTexture;
    private int[] mGlTextures = null;
    private Object mFrameSyncObject = new Object();
    private boolean mFrameAvailable = false;

    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

    public void startCamera(View v) {
        try {

            this.initCamera(0);
            this.StartCamera();


        } catch (Throwable throwable) {
            throwable.printStackTrace();
        }
    }

    private void StartCamera() {

        try {

            mCamera.startPreview();

            long startWhen = System.nanoTime();
            long desiredEnd = startWhen + DURATION_SEC * 1000000000L;

            int frameCount = 0;

            while (System.nanoTime() < desiredEnd) {
// Feed any pending encoder output into the muxer.

                awaitNewImage();
            }
        } finally {
// release everything we grabbed
            releaseCamera();

        }
    }

    /**
     * Stops camera preview, and releases the camera to the system.
     */
    private void releaseCamera() {
        if (VERBOSE) Log.d(TAG, "releasing camera");
        if (mCamera != null) {
            mCamera.stopPreview();
            mCamera.release();
            mCamera = null;
        }
    }

    private void initCamera(int cameraId) {

        mCamera = Camera.open(cameraId);
        if (mCamera == null) {
            Log.d(TAG, "No front-facing camera found; opening default");
            mCamera = Camera.open();    // opens first back-facing camera
        }
        if (mCamera == null) {
            throw new RuntimeException("Unable to open camera");
        }

        Camera.Parameters parms = mCamera.getParameters();
        parms.setPreviewSize(640, 480);
        mGlTextures = new int[1];
        GLES20.glGenTextures(1, mGlTextures, 0);


        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, mGlTextures[0]);


        GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER,
                GLES20.GL_NEAREST);
        GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER,
                GLES20.GL_LINEAR);
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S,
                GLES20.GL_CLAMP_TO_EDGE);
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T,
                GLES20.GL_CLAMP_TO_EDGE);
        mSurfaceTexture = new SurfaceTexture(mGlTextures[0]);
        try {
            mCamera.setPreviewTexture(mSurfaceTexture);
        } catch (IOException e) {
// TODO Auto-generated catch block
            e.printStackTrace();
        }
        mSurfaceTexture.setOnFrameAvailableListener(MainActivity.this);


    }

    public void awaitNewImage() {
        final int TIMEOUT_MS = 4500;
        synchronized (mFrameSyncObject) {
            while (!mFrameAvailable) {
                try {
// Wait for onFrameAvailable() to signal us.  Use a timeout to avoid
// stalling the test if it doesn't arrive.
                    if (VERBOSE) Log.i(TAG, "Waiting for Frame in Thread");
                    mFrameSyncObject.wait(TIMEOUT_MS);
                    if (!mFrameAvailable) {
// TODO: if "spurious wakeup", continue while loop
                        throw new RuntimeException("Camera frame wait timed out");
                    }
                } catch (InterruptedException ie) {
// shouldn't happen
                    throw new RuntimeException(ie);
                }
            }
            mFrameAvailable = false;
        }


    }

    @Override
    public void onFrameAvailable(SurfaceTexture st) {
        if (VERBOSE) Log.d(TAG, "new frame available");
        synchronized (mFrameSyncObject) {
            if (mFrameAvailable) {
                throw new RuntimeException("mFrameAvailable already set, frame could be dropped");
            }
            mFrameAvailable = true;
            mFrameSyncObject.notifyAll();
        }
    }
}

I am also facing the same issue. Did you by chance find a solution? — Subhransu, Dec 09 '13 at 14:59
In my understanding onFrameAvailable should be used with thread. With that i am not facing the issue — Nehal Shah, Dec 11 '13 at 03:53

score 1 · Answer 1 · edited Oct 27 '16 at 13:36

1

I think you have to call SurfaceTeture.updateTextImage() after your OnFrameAvailable() Callback to tell the camera "I've used your last frame, give me another one".

(Sorry but my English cannot provide a better explanation)

edited Oct 27 '16 at 13:36

Yury Fedorov

14,508
6
50
66

answered Oct 25 '16 at 19:28

cescom

116
6

score 0 · Answer 2 · answered Sep 25 '17 at 12:54

0

    @Override   
    public void onFrameAvailable(SurfaceTexture surfaceTexture) {
        ...
        surfaceTexture.updateTexImage();
    }

had the same problem, seems like I forgot to call for the updateTexImage()

answered Sep 25 '17 at 12:54

Ariel Yust

585
4
11

You should not call updateTexImage() on anything but the OpenGL thread (e.g. during rendering). https://developer.android.com/reference/android/graphics/SurfaceTexture.html – Rupert Rawnsley Jan 10 '18 at 09:37
@RupertRawnsley `onFrameAvailable()` IS RUN on OpenGL thread. once you make the Camera listen to SurfaceTexture changes then you receive these changes into this callback on an OpenGL thread, you can safely call `surfaceTexture.updateTexImage();` and draw anything on your EglSurfaces. check [google/grafika/][1] for examples. [1]: https://github.com/google/grafika – Ariel Yust Jan 18 '18 at 08:00
[The documentation](https://developer.android.com/reference/android/hardware/Camera.html#setPreviewTexture\(android.graphics.SurfaceTexture\)) makes no mention of that. Also none of the three calls to onFrameAvailable in the grafika examples use the GL thread and in fact the comments for [this implementation](https://github.com/google/grafika/blob/bc29f6715f25ff841f96f6faa9ccd12d89d9ef97/app/src/main/java/com/android/grafika/CameraCaptureActivity.java#L397) say you need to call updateTexImage on the surface owner thread (GL in this case). – Rupert Rawnsley Jan 19 '18 at 11:08
@RupertRawnsley the documentation fails to mention many things, actually Grafika IS using Gl-Thread, it uses the UI-Thread which also acts as a GL-Thread, in any case you can call for `updateTexImage ()` inside `onFrameAvailable()` if you `offscreenSurface.makeCurrent()` before opening the camera and setting the `onFrameAvailable` listener. Please don't believe me, just try it and see that it works :) – Ariel Yust Jan 21 '18 at 19:44

score 0 · Answer 3 · answered Dec 05 '21 at 12:59

use method setOnFrameAvailableListener(@Nullable final OnFrameAvailableListener listener, @Nullable Handler handler) replace setOnFrameAvailableListener(@Nullable OnFrameAvailableListener listener).

in your case, you can modify the code as:

frameUpdateThread = new HandlerThread("frameUpdateThread");
frameUpdateThread.start();
mSurfaceTexture.setOnFrameAvailableListener(MainActivity.this, Handler(frameUpdateThread.getLooper()));

score -1 · Answer 4 · answered Dec 13 '13 at 12:30

-1

In my understanding onFrameAvailable should be used with thread. With that i am not facing the issue and also make sure updatetextImage is called after receiving the frames

answered Dec 13 '13 at 12:30

Nehal Shah

93
2
11

OnFrameAvailable callback is not coming

4 Answers4