7

is it possible to share a SurfaceTexture between two processes, for example an Activity and a Service?

I would like to create an Activity with a TextureView, and update its SurfaceTexture from a separate Service.

So far I'm creating a SurfaceTexture with a generated OpenGL texture (through glGenTextures), then I set this surface texture to my TextureView:

mSurfaceTexture = new SurfaceTexture(texture_id);
mTextureView.setSurfaceTexture(mSurfaceTexture);

Displaying the camera preview to this SurfaceTexture on the activity is working fine:

mCamera = Camera.open();
mCamera.setPreviewTexture(mTextureView.getSurfaceTexture());
mCamera.startPreview();

I would like to do the same but from a separate service, passing the texture_id to it, something like:

mCamera = Camera.open();
mCamera.setPreviewTexture(new SurfaceTexture(texture_id));
mCamera.startPreview();

The reason is that I have a separate process calling a private api that needs a SurfaceTexture to stream some content, and this process is accessed through aidl from the application.

Thanks

Philippe Goncalves
  • 193
  • 2
  • 7

1 Answers1

8

The system supports doing what you want, but I don't know if it's possible with the current public API.

Some background to ensure we're on the same page...

A Surface is the producer side of a producer-consumer pair. With SurfaceTexture, the app has access to both ends. Anything rendered to the SurfaceTexture's Surface is converted to an OpenGL ES "external" texture.

The way that the media and display APIs work is by having the consumer create the pair and then hand the Surface to the app. This is why, if you create a SurfaceView, you can't use the Surface until the surfaceCreated() callback fires -- the BufferQueue object is created by the system graphics compositor (SurfaceFlinger), and the producer side is passed through Binder IPC to your app. Same story for Surface input to MediaCodec encoders, which are created by the mediaserver process.

You can pass the Surface you get from SurfaceView or SurfaceTexture to something that produces output, such as Camera preview or a MediaCodec decoder. Under the hood, these pass the Surface through IPC to the mediaserver process. As frames are produced, the graphic buffers are passed by reference to the consumer. SurfaceFlinger displays the frames it gets from SurfaceView, SurfaceTexture just converts them to textures.

So what you'd want to do is create a SurfaceTexture in the app, construct a Surface for it, and pass that Surface to your service. Your service would generate frames and write them to the Surface, which would send them through IPC to the SurfaceTexture consumer in your app, which would convert them into GLES textures.

I haven't tried passing a Surface through AIDL, so I don't know if it'll actually work or not.

Passing a texture ID between processes won't work. You can access a texture from two different EGLContexts if you pass the first context as an argument when creating the second one, but I don't think that's expected to work cross-process.

More details on the way the system works can be found in the architecture doc.

fadden
  • 51,356
  • 5
  • 116
  • 166