constraints: My system has 250MB RAM memory available. The amount of data generated by the producer lies around 5Mb/sec. I need to provide a buffer of around 30 seconds. So around 150MB...
What I'm trying to achieve: Implement a single-producer single-consumer ringbuffer in a tmpfs mount. One thread should work as a producer filling the buffer and multiple threads should fetch data from the buffer.
So far I've tried vectors (c++) all leading to an allocation error. It seems using contiguous memory allocation won't work (for amounts around 50 MB). Using deque (non contiguous) on the other hand works fine. But I'm limited to the kernel response regarding memory management and the available memory could lie below the expected amount to initialize the buffer... That's why I would rather use tmpfs as it is mounted with a predefined size.
Using a class containing a circular buffer with relevant concurrently-safe methods and properties (possibly lock-free?), I could create the buffer using a sequence of files in tmpfs with a duration of 1 second/each.
Would this be fast enough compared with the deque implementation or would I feel the lag caused by accessing the files concurrently from multiple threads?
What are your thoughts guys...Am I missing something?...
