Transmit output of MediaProjection API over network

Question

I've been reading on MediaProjection API, and checked out a few sample projects on Github that show how to use it.
All those projects only show how to save the screen capture in a file ( record the screen ), and nothing about transmiting it in real-time to another device ( screencast ).

I wasn't sure that this was possible via this API, but then I read the docs :

Screen capturing and sharing

Android 5.0 lets you add screen capturing and screen sharing capabilities to your app with the new android.media.projection APIs. This functionality is useful, for example, if you want to enable screen sharing in a video conferencing app.

The new createVirtualDisplay() method allows your app to capture the contents of the main screen (the default display) into a Surface object, which your app can then send across the network. The API only allows capturing non-secure screen content, and not system audio. To begin screen capturing, your app must first request the user’s permission by launching a screen capture dialog using an Intent obtained through the createScreenCaptureIntent() method.

For an example of how to use the new APIs, see the MediaProjectionDemo class in the sample project.

I've searched the whole day for a sample on how to use the Surface object to transmit it's data, then receive and show them on another device, but with no luck.

So the question is, how can it be done ? Can anyone point me in the right direction ?

Hi @rick-sanchez, Did you manage to find any answer your question ? — thomasbabuj, Nov 28 '16 at 09:27
@thomasbabuj No i didn't find anything useful. But was able to add the same functionality using WebRTC — Rick Sanchez, Nov 28 '16 at 09:49
wow, that's great. can you share some reference URL related WebRTC and MediaProjection API — thomasbabuj, Nov 28 '16 at 10:34
Follow the instructions on https://webrtc.org/native-code/ about building the native libraries. Then get the sample app AppRtcMobile (it is in the webrtc repository) in which you can see how tie in the MediaProjection API with WebRTC. The relevant class for this is the ScreenCapturerAndroid. — Rick Sanchez, Nov 28 '16 at 11:35
@Vivek No. I ended up using the webbrtc API that I mentioned in last comment — Rick Sanchez, Apr 17 '18 at 05:13
@RickSanchez do you have any idea how can integrate webbrtc API in kurento android native app ? — Vivek Patel, Apr 17 '18 at 07:15

score 1 · Accepted Answer · answered Jan 23 '19 at 13:57

Well, you can extract the frames from the surface you pass to the createVirtualDisplay() (with ImageReader which is one of the simplest ways to do that or by using SurfaceTexture, depending on your needs), after you get a bitmap, you just send it over the network (that part is application specific).

For instance you could establish a websocket connection to your server and use it to transmit the frames (that's what I did when I played with the API), alternatively you can use WebRTC for that.

Transmit output of MediaProjection API over network

1 Answers1