Write a CircularBuffer (swift-nio) to a file in Swift using OutputStream?

Question

I am trying to use a CircularBuffer<UInt8> from SwiftNIO to store data and once the buffer is almost full dump the contents to a file using an OutputStream. Unfortunately, the OutputStream.write() method takes UnsafePointer as an argument, while the CircularBuffer can output UnsafeBufferPointer . Is there a way to convert CircularBuffer to UnsafePointer?

I have tried to extend CircularBuffer with the following code that I am using with success to convert structs to Byte arrays as it was suggested that CircularBuffer is in fact a struct, but I am getting garbage in my output file:

extension CircularBuffer {
    func toBytes() -> [UInt8] {
        let capacity = MemoryLayout<Self>.size
        var mutableValue = self
        return withUnsafePointer(to: &mutableValue) {
            return $0.withMemoryRebound(to: UInt8.self, capacity: capacity) {
                return Array(UnsafeBufferPointer(start: $0, count: capacity))
            }
        }
    }
}

Any thoughts?

If you have an UnsafeBufferPointer then you can take its `.baseAddress` to get an UnsafePointer. — Martin R, Sep 29 '20 at 19:44
Duplicate of https://stackoverflow.com/questions/64120725/how-can-i-dump-a-circuralbuffer-swift-nio-to-a-file-in-swift ? You don't get to evade duplicate-ness by asking the same question again. — matt, Sep 29 '20 at 19:50
While I agree that a question should be improved (instead of deleted and asked again) I also think that the other question has been wrongly closed as a duplicate. This is not about writing a struct to a file, but about writing the “contents” of that circular buffer structure to a file. — Martin R, Sep 29 '20 at 19:59

Martin R · Accepted Answer · 2020-09-29T21:45:07.797

CircularBuffer is a struct with an internal ContiguousArray for the element storage. ContiguousArray is also a struct, with internal pointers to the actual element storage.

Your current code produce garbage because it returns the memory representation of the struct CircularBuffer itself, and not the bytes of the elements which it represents.

As a collection, CircularBuffer has a withContiguousStorageIfAvailable() method which calls a closure with a pointer to the element storage if such contiguous storage exists. The closure is called with an UnsafeBufferPointer argument from which you can obtain the baseAddress:

var buffer: CircularBuffer<UInt8> = ...
let os: OutputStream = ...
// ...
let amountWritten =  buffer.withContiguousStorageIfAvailable {
    os.write($0.baseAddress!, maxLength: $0.count)
}

But there is a problem: CircularBuffer just inherits the default implementation from Sequence which returns nil without calling the closure. This is a known issue. So the above code would compile, but not work.

A simple way (at the cost of copying the contents) would be to use that you can initialize an array from a collection:

var buffer: CircularBuffer<UInt8> = ...
let os: OutputStream = ...
// ...
let contents = Array(buffer)
let amountWritten = os.write(contents, maxLength: contents.count)

So at the moment, there is no option other than copying the contents to an Array and then outputting them :( Thx for your reply. — jiko, Sep 29 '20 at 21:53
@jiko: The internal storage is an array of `Element?`, i.e. of optionals, so there actually is no pointer to a contiguous storage of `Element`s which could be passed to the `write` method (this is discussed in the GitHub issue that I linked to). So you have to extract the elements somehow. Either all together as an array, or one by one as suggested by Matt. — Martin R, Sep 29 '20 at 21:59

score 1 · Answer 2 · answered Sep 29 '20 at 20:48

You can also cycle through the elements of the buffer one at a time, though that would probably be quite inefficient:

    var cb = CircularBuffer<UInt8>()
    cb.append(contentsOf:[1,2,3])
    cb.append(contentsOf:[4,5,6])
    let stream = OutputStream.toMemory()
    stream.open()
    for var i in cb {
        stream.write(&i, maxLength:1)
        print(i)
    }
    stream.close()
    print(stream.property(forKey: .dataWrittenToMemoryStreamKey) as Any)

Write a CircularBuffer (swift-nio) to a file in Swift using OutputStream?

2 Answers2