circular buffer implementation

Question

Yes again, I come again with that very straight forward implementation which is something like this:

    // write data always! if buffer is already full, overwrite old data!
    void Put( const CONTENT_TYPE &data )
    {
        buffer[ inOffset++] = data;
        inOffset%=size;

        // was data overwritten, skip it by increment read offset
        if ( inOffset == outOffset ) 
        {
            outOffset++;
            outOffset%=size;
            std::cout << "Overwrite" << std::endl;
        }
    }

    CONTENT_TYPE Pull()
    {
        CONTENT_TYPE data = buffer[ outOffset++ ];
        outOffset %= size;
        return data;
    }

But this simple algorithm utilizes only size-1 one elements of the buffer!

If I want to avoid that, I only found a solution with adding another counter variable, which wastes me sizeof(counter_var) - sizeof(element) bytes.

Q: Is there a solution which did not waste memory? It looks so terrible simple but I can't catch it :-)

Remark: There are some more lines of code to protect for empty reads and other stuff, but this is not important to the question. And it is not tagged c++ because the algorithm did not depend on the language, also if I give a c++ code example.

Your first sentence seems to indicate this is an *answer* to something rather than a question; maybe you'd like to rephrase this; also, language-tagging is always a good idea. — Marcus Müller, Aug 10 '16 at 14:46
@MarcusMüller: changed the first sentence, language is not important for the algo here. — Klaus, Aug 10 '16 at 14:50
I think you can solve this by replacing one of the offset variables by a size variable. — Nicolas, Aug 10 '16 at 14:53
@klaus I'm still not 100% sure I understand the *problem* you're trying to solve. Could you state more clearly what you actually want to implement? — Marcus Müller, Aug 10 '16 at 14:58
@MarcusMüller: My implementation only utilizes size-1 elements in the buffer. Given a buffer size of 8 you only can store 7 elements. If you add a counter var, the overall size of buffer + management data increases by sizeof(counter) but utilizes one more element which results in overall additional cost sizeof(counter_var) - sizeof ( element ). If sizeof(element) is smaller than sizeof(counter_var) this is not useful. I search a solution which comes without additional cost in space and hopefully also in runtime consumption. — Klaus, Aug 10 '16 at 15:10
I mean your question title says "circular buffer", and you seem to be perfectly capable of using the modulo operator to index an array, but I seem to be missing something, because with that, you'd use all elements of that array. — Marcus Müller, Aug 10 '16 at 15:15
@MarcusMüller this is a perfectly reasonable implementation of a circular buffer; when the two offsets are equal, the buffer could be either full or empty. It's generally empty by convention, meaning there's no way to mark the buffer as full. — Mark Ransom, Aug 10 '16 at 15:17
@MarkRansom absolutely! But we both then don't see why he's using one element less, right? — Marcus Müller, Aug 10 '16 at 15:20
@MarcusMüller: If you use the given implementation you will see that a buffer of size 8 can only hold 7 elements at a time but all slots of the buffer will be used circular. But as already mentioned there is always 1 element gap on a "full" buffer. — Klaus, Aug 11 '16 at 06:44
@Klaus are you the one that downvoted my answer? Did you try it? — Mark Ransom, Aug 11 '16 at 13:14
@MarkRansom: No, why do you think this? I prefer giving comments to improve maybe wrong answers. But I have not inspected your code in the moment so that there is no reason for commenting nor voting. — Klaus, Aug 11 '16 at 13:22
I prefer it that way too, I was just wondering if I had made a mistake and somebody wasn't kind enough to point it out. At least @MarcusMüller was honest when he did it to me yesterday. — Mark Ransom, Aug 11 '16 at 13:38

Gene · Answer 1 · 2016-08-12T02:45:34.433

You can use two integers and fill all slots if one is an index and the other an element count, then convert to find the second index on the fly:

void put(const ELEMENT& element) {
  if (nElements == size) throw "put: buffer full";
  buffer[(start + nElements++) % size] = element;
}

ELEMENT get() {
  if (nElements == 0) throw "get: buffer empty";
  ELEMENT& value = buffer[start];
  start = (start + 1) % size;
  --nElements;
  return value;
}

Of course you can replace the mod operations with if (foo > size) foo -= size; if you like.

Marcus Müller · Answer 2 · 2016-08-10T16:23:42.457

0

You'd just deal with that by using different points in time at which you do the modulo operation; assume we increase the read and write pointers after every access. If we now do the read pointer's modulo instantly after increasing, and the write pointer's modulo just right before reading, the |write-read| of a full buffer would be the length of the buffer, without any special case handling. For that to work, your write pointer should always be used % buffer_length, but stored % (2 * buffer_length).

I don't especially like Mark's answer, because handling things as special cases is usually not a good idea, as little as introducing negative sentinel values is in a place where you'd typically used size_t (i.e. an unsigned integer).

edited Aug 10 '16 at 16:23

answered Aug 10 '16 at 15:24

Marcus Müller

34,677
4
53
94

That's not the way a circular buffer works. You only increment one offset or the other, depending on whether you're reading or writing; the only exception is when you overwrite because the buffer was full, which requires modifying the other offset to erase the oldest element. In your plan how do you tell the difference between an empty and a full buffer? P.S. I don't like my answer either, but it's the best I can come up with that meets the constraints - it's much easier to just accept the loss of 1 element capacity. – Mark Ransom Aug 10 '16 at 15:29
exactly, you're increasing either read or write pointer. By using different semantics when to "reset" to zero, you keep the full/empty information contained in the read-write pointer difference. Empty buffer: read==write pointer; full buffer read != write, but (read + buff length )% buff length = write – Marcus Müller Aug 10 '16 at 16:20
ah, I should add that write buffer exists on twice the buffer length, and just gets aliased onto the single length. – Marcus Müller Aug 10 '16 at 16:22
The devil's in the details. If you write length*2 into the buffer, the write index will wrap again and it looks empty once more. – Mark Ransom Aug 10 '16 at 18:15

Mark Ransom · Answer 3 · 2016-08-11T13:12:30.807

-1

You could use a special sentinel value for one of the offsets, such as -1, to indicate that the buffer is full or empty. This will complicate your code for checking and modifying the offset.

// write data always! if buffer is already full, overwrite old data!
void Put( const CONTENT_TYPE &data )
{
    buffer[ inOffset++] = data;
    inOffset%=size;

    // was data overwritten, skip it by setting read offset to sentinel
    if ( inOffset == outOffset || outOffset == -1 ) 
    {
        outOffset = -1;
        std::cout << "Overwrite" << std::endl;
    }
}

CONTENT_TYPE Pull()
{
    if (outOffset == -1)
        outOffset = inOffset;
    CONTENT_TYPE data = buffer[ outOffset++ ];
    outOffset %= size;
    return data;
}

bool IsEmpty()
{
    return outOffset == inOffset;
}

edited Aug 11 '16 at 13:12

answered Aug 10 '16 at 15:20

Mark Ransom

299,747
42
398
622

(You might want to point out _the_ saving grace of using _sentinel values_.) – greybeard Aug 11 '16 at 22:16
@greybeard not sure there is one. This is *almost* but not quite lock free if you have different threads calling `Put` and `Pull`. – Mark Ransom Aug 11 '16 at 23:02

circular buffer implementation

3 Answers3