How Go buffer read data from io.Reader?

Question

I'm learning for Go bufio package, and confused by a code snippet below:

1  s1 := strings.NewReader(strings.Repeat("a", 16) + strings.Repeat("b", 16))
2  r := bufio.NewReaderSize(s1, 16)
3  b, _ := r.Peek(3)
4  fmt.Printf("%q\n", b)
5  r.Read(make([]byte, 16))
6  r.Read(make([]byte, 15))
7  fmt.Printf("%q\n", b)

// "aaa"
// "bbb"

Isn't that r in line 2 an empty []byte? Why can user r.Peek(3) to figure out the "aaa" result?

Assume the bufio.NewReaderSize(s1, 16) can Read 16 bytes from Reader s1, this make line 3 reasonable; Why use twice r.Read() in line5 and line6?

And Isn't that the undelying array of r in line 5 is "aaaaaaaaaaaaaaaa", and line6 became the "bbbbbbbbbbbbbbb"?

Or maybe the underlying array of r always be "aaaaaaaaaaaaaaaabbbbbbbbbbbbbbbb"?

If you can give me any inspiration, Thank you for your help.

My mistake, I think the thing seems like this: line 2 create a buffer Reader (`*bufio.Reader`), when call `r.Peek(3)`, `r` read 3 bytes from s1, but Peek will not real consume the s1 data; so `b := "aaa"` ; and `r.Read(make([]byte, 16))` consumed "aaaaaaaaaaaaaaaa" from s1, the underlying array of `buf []byte` becames "aaaaaaaaaaaaaaaa", and the underlying of `b` from `r.Peek(3)` same with `buf []byte` field, so `b = aaa` , when second call `r.Read(make([]byte, 15))` in line 6, the underlying array of `buf []byte` is "bbbbbbbbbbbbbbb" , and underlying arr of `b` changed, so `b = "bbb"` — AchillesHeel, Jun 08 '22 at 03:57
also note : the documentation for [`.Peek()`](https://pkg.go.dev/bufio#Reader.Peek) states : "The bytes stop being valid at the next read call." So you shouldn't reaccess `b` after having called `.Read()`. It won't trigger any invalid memory access bug (go is safe in that regard), it's just that depending on how the buffer is implemented, you really have no control over what `b` will contain at line 7 (for starters : just try to change the underlying size of your `bufio.Reader`) — LeGEC, Jun 08 '22 at 05:30

score 0 · Answer 1 · answered Jun 10 '22 at 14:36

Isn't that r in line 2 an empty []byte? Why can user r.Peek(3) to figure out the "aaa" result?

If the underlying buffer is empty or has not enough space left to satisfy Peek(n), bufio.Reader will call its underlying io.Reader Read method to fill the buffer, you can check this by extending the returned slice from Peek, b , _ := r.Peek(1); fmt.Printf("%q\n", b[:cap(b)]).

The problem is that you're misusing the package, there is no guarantee nor API to get the underlying array or even that it's used, Peek returns the underlying array in a slice only to avoid a copy/memory allocation, in fact bufio may just ignore buffering and call the underlying reader directly, see: https://go.dev/play/p/WT6AUfEno3t

As noted in Peek documentation, the returned slice from Peek is only valid values until a call to Read, to correctly get data from Reader, use the Read method, not tricks to get its underlying buffer.

How Go buffer read data from io.Reader?

1 Answers1