0

I wrote test application for capturing the images with MediaProjection class.

imageReader = ImageReader.newInstance(currentWidth, currentHeight, PixelFormat.RGBA_8888, 2);
imageReader.setOnImageAvailableListener(this, null);

virtualDisplay = mediaProjection.createVirtualDisplay("captureDisplay",
                    currentWidth, currentHeight,  DisplayManager.VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY |
                            DisplayManager.VIRTUAL_DISPLAY_FLAG_PUBLIC| 
                            DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR |
                            DisplayManager.VIRTUAL_DISPLAY_FLAG_SECURE |
                          DisplayManager.VIRTUAL_DISPLAY_FLAG_PRESENTATION, Screen.getDensity(mContext),                          imageReader.getSurface(), null, null);

// DisplayManager flags are trails only

and in onImageAvailable(ImageReader reader) method

i tried to get the image as follows:

Bitmap bitmap;

Image mImage = null;

try {

mImage = mImageReader .acquireLatestImage();

if (img != null) {

    final Image.Plane[] planes = mImage.getPlanes();

    final ByteBuffer buffer = planes[0].getBuffer();

    int pixelStride = planes[0].getPixelStride();

    int rowStride = planes[0].getRowStride();

    int rowPadding = rowStride - pixelStride * mImage.getWidth();

    bitmap = Bitmap.createBitmap(mImage.getWidth() + rowPadding/pixelStride, mImage.getHeight(), Bitmap.Config.ARGB_8888);  

    bitmap.copyPixelsFromBuffer(buffer);

    ByteArrayOutputStream stream = new ByteArrayOutputStream();

    bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);

   byte[] rawData = lastImageAcquiredRaw = stream.toByteArray();

   if (rawData != null) {

    Bitmap fullScreen = BitmapFactory.decodeByteArray(rawData, 0, rawData.length);

    savebitmap(fullScreen,"image",i); //Saving bitmap in storage

    }

}

Till now I am OK and I am getting correct Image when my app is in landscape orientation. Problem facing is on orientation change, i am not getting proper images. Some times again on changing back to landscape also not capturing properly.

I gone through ImageReader.java class. Nothing has mentioned like need to handle in orientation changes.

Tried using acquireNextImageNoThrowISE(), acquireNextImage() but no use.

Did any one tried or having possibility to get proper image in orientation?

Please help me in getting proper image.

CopsOnRoad
  • 237,138
  • 77
  • 654
  • 440
user2318724
  • 259
  • 5
  • 16

1 Answers1

1

I know this post is a bit old, but the last couple of days I am strangling with the same thing. I am also using MediaProjection API, VirtualDisplay and ImageReader's surface. The solution I am going to demonstrate is not 100% full-proof, so if anyone has any useful suggestions is more than welcome.

Step one: add an OrientationChangeCallback after you create Virtual Display

OrientationChangeCallback orientationChangeCallback = new OrientationChangeCallback(context);
if (orientationChangeCallback.canDetectOrientation()) {
  orientationChangeCallback.enable();
}

Step two: define the orientation callback, which in essence restarts the capture

private class OrientationChangeCallback extends OrientationEventListener {
        OrientationChangeCallback(Context context) {
            super(context);
        }

        @Override
        public void onOrientationChanged(int orientation) {
            final int rotation = mDisplay.getRotation();
            if(rotation != mRotation) {
                mRotation = rotation;
                if (mVirtualDisplay != null) {
                    mVirtualDisplay.release();
                }
                if (mImageReader != null) {
                    mImageReader.setOnImageAvailableListener(null, null);
                }
                if (!mProjectionStopped) {
                    createVirtualDisplay();
                }
            }
        }
    }

Note that I am holding references to ImageReader and VirtualDisplay, and also have a volatile boolean mRotation to check if the screen actually rotated.

The problem I have is that on some devices (Nexus 5X included) the image can become corrupt after X number or rotations (usually 10-20). If I rotate the device one more time the image is captured correctly. On quick devices (Google Pixel) I cannot replicate the corrupt image issue, so I am guessing it might be a synchronisation issue.

mtsahakis
  • 793
  • 1
  • 8
  • 18