Data race issues while reading data from file and sending it simultaneously

Question

I am trying to read data from a file and send it immediately the chunk read without waiting the other goroutine to complete file read. I have two function

func ReadFile(stream chan []byte, stop chan bool) {
    file.Lock()
    defer file.Unlock()
    dir, _ := os.Getwd()
    file, _ := os.Open(dir + "/someFile.json")
    chunk := make([]byte, 512*4)
    for {
        bytesRead, err := file.Read(chunk)
        if err == io.EOF {
            break
        }
        if err != nil {
            panic(err)
            break
        }
        stream <- chunk[:bytesRead]
    }

    stop <- true
}

First one is responsible for reading the file and sending data chunks to the "stream" channel received from the other function

func SteamFile() {
    stream := make(chan []byte)
    stop := make(chan bool)

    go ReadFile(stream, stop)

L:
    for {
        select {
        case data := <-stream:
            //want to send data chunk by socket here
            fmt.Println(data)
        case <-stop:
            break L
        }
    }
}

Second function creates 2 channel, sends them to the first function and starts to listen to the channels. The problem is that sometimes data := <-stream is missing. I do not receive for example the first part of the file but receive all the others. When I run the program with -race flag it tells that there is a data race and two goroutines write to and read from the channel simultaneously. To tell the truth I thought that's the normal way to use channels but now I see that it brings to even more issues. Can someone please help me to solve this issue?

score 1 · Accepted Answer · answered Nov 24 '20 at 10:18

1

When I run the program with -race flag it tells that there is a data race and two goroutines write to and read from the channel simultaneously. To tell the truth I thought that's the normal way do use channels.

It is, and you are almost certainly misunderstanding the output of the race detector.

You're sharing the slice in ReadFile (and therefore it's underlying array), so the data is overwritten in ReadFile while it's being read in SteamFile [sic]. While this shouldn't trigger the race detector it is definitely a bug. Create a new slice for each Read call:

  func ReadFile(stream chan []byte, stop chan bool) {
      file.Lock()
      defer file.Unlock()
      dir, _ := os.Getwd()
      file, _ := os.Open(dir + "/someFile.json")
-     chunk := make([]byte, 512*4)
      for {
+         chunk := make([]byte, 512*4)
          bytesRead, err := file.Read(chunk)

answered Nov 24 '20 at 10:18

Peter

29,454
5
48
60

You were right, I thought that writing data to the channel it makes copy just like passing the argument to the function. As the race detector was bringing the error on reading channel I thought that the issue was in a channel. Creating new slice inside the loop solved my problem – sendan Nov 24 '20 at 12:11
I also wonder, will OP experience any disadvantage in performance by using an unbuffered channel (`make(chan []byte)`)? – Zyl Nov 24 '20 at 12:11
@vladazn, it does copy the slice (just like it does for a function call indeed), but not the underlying array. If the difference isn't clear, take a look at [Go Slices: usage and internals](https://blog.golang.org/slices-intro). – Peter Nov 24 '20 at 16:19

Data race issues while reading data from file and sending it simultaneously

1 Answers1