2

I'm having a bit of trouble thinking of an efficient solution. There are a few problems I am foreseeing, the first being...

OOM Prevention

If I wanted the past 30 seconds or even 5 minutes it's doable, but what if I wanted the past 30 minutes or full hour, or maybe EVERYTHING? Keeping a byte buffer means storing it in RAM. Storing over a hundred megabytes sounds like Virtual Memory suicide.

Okay so what if we store a Y amount of time, say 30 seconds, of the previously recorded media to disk in some tmp file. That potentially could work and I can use a library like mp4 parser to concatenate them all when finished. However...

If we have 30 minutes worth that's about 60 30-second clips. This seems like a great way to burn through an SD card and even if that's not a problem I can't imagine the time needed to concatenate over a hundred files into one.

From what I've been researching, I was thinking of using local sockets to do something like...

MediaRecorder -> setOutputFile(LocalSocket.getFD())

Then in the local socket...

LocalSocket -> FileOutputStream -> write(data, position, bufsiz) -> flush()

Where the background thread handles writing and keeping track of the position, and the buffer.

This is purely pseudocode and I'm not far enough in yet to test this, am I going in the right direction with this? From what I'm thinking, this only keeps one file which gets overwritten. As it only gets written to once every Y seconds it minimized IO overhead and also minimizes the amount of RAM it eats up.

Video Length to Buffer Size

How would I obtain the size the buffer should be from requested video size. It's strange since I see some long videos that are small but short videos that are huge. So I don't know how to accurately determine this. Anyone know how I can predict this if I know the video length, encoding, etc which gets set up from Media Recorder?

Examples

Does anyone know of any examples of this? I don't think the idea is entirely original but I don't see a lot of them out there and if it does it is closed source. An example goes a long way.

Thanks in advance

fadden
  • 51,356
  • 5
  • 116
  • 166
Louis Jenkins
  • 391
  • 1
  • 2
  • 10

1 Answers1

3

The "continuous capture" Activity in Grafika does this, using MediaCodec rather than MediaRecorder. It keeps the last N seconds of video in a circular buffer in memory, and writes it to disk when requested by the user.

The CircularEncoder constructor estimates the memory requirements based on target bit rate. At a reasonable rate (say 4Mbps) you'd need 1.8GB to store an hour worth of video, so that's not going to fit in RAM on current devices. Five minutes is 150MB, which is pushing the bounds of good manners. Spooling out to a file on disk is probably necessary.

Passing data through a socket doesn't buy you anything that you don't get from an appropriate java.util.concurrent data structure. You're just involving the OS in a data copy.

One approach would be to create a memory-mapped file, and just treat it the same way CircularEncoder treats its circular buffer. In Grafika, the frame data goes into a single large byte buffer, and the meta-data (which tells you things like where each packet starts and ends) sits in a parallel array. You could store the frame data on disk, and keep the meta-data in memory. Memory mapping would work for the five-minute case, but generally not for the full hour case, as getting a contiguous virtual address range that large can be problematic.

Without memory-mapped I/O the approach is essentially the same, but you have to seek/read/write with file I/O calls. Again, keep the frame metadata in memory.

An additional buffer stage might be necessary if the disk I/O stalls. When writing video data through MediaMuxer I've seen periodic one-second stalls, which is more buffering than MediaCodec has, leading to dropped frames. You can defer solving that until you're sure you actually have a problem though.

There are some additional details you need to consider, like dropping frames at the start to ensure your video starts on a sync frame, but you can see how Grafika solved those.

fadden
  • 51,356
  • 5
  • 116
  • 166
  • Okay wow definitely did not expect a reply so detailed. I can't explicitly comment on the content right now but I have to thank you for the amount of effort. I'll read later and get back to your post. – Louis Jenkins Nov 23 '15 at 20:00
  • Just commenting to say that I didn't forget this answer, however as of right now I have other things needing to be implemented first BEFORE I can work on this implementation,I have do have a few base questions though: 1) Can I presumably use the surface obtained MediaProjection directly with MediaRecorder? If so, is it a drop-in replacement? 2) Would it yield better results and/or performance if I did it with JNI/NDK? I am familiar with mmap and C's low-level read-writes, would this help? I've never done JNI/NDK before though, but I always wanted an excuse to learn it. – Louis Jenkins Nov 24 '15 at 07:31
  • (1) MediaCodec and MediaRecorder provide a Surface when asked. You pass that to MediaProjection's `createVirtualDisplay()` call. Works the same for either of those, or SurfaceView for that matter. (2) You're doing very little computation in your code. Your app is just directing traffic, so there's not much value in using the NDK. There might be an advantage to using native I/O if the Java-language equivalents don't fit as well, potentially allowing you to avoid an extra copy, but I'd strongly recommend getting it to work first without the NDK. – fadden Nov 24 '15 at 16:27
  • For giving me a valid direction and advice on how to pursue the answer to my question, I'll award yours best answer (as it also is the only one). Thanks once again – Louis Jenkins Nov 24 '15 at 18:44
  • I have another question, can I change the state of MediaCodec/MediaRecorder and others from another thread safely? – Louis Jenkins Nov 25 '15 at 15:32
  • Unless it explicitly states otherwise, assume that objects should only be used from one thread at a time. It's usually best to create the thread, then have that thread create and configure the objects. Establish a Handler on the thread to receive state change requests. Most of the examples in Grafika use a dedicated renderer and/or encoder thread. – fadden Nov 25 '15 at 16:27