idiomatic way to use c style iterator with Go

Question

I'm very new to Go programming (3-4 days), and I'm trying to write some code that reads a binary file using an existing third-party C library using cgo. The C library's way of doing this seems fairly standard (for C). Slightly simplified it looks like:

int main(int argc, char *argv[]) {
    file_t *file = file_open(filename);
    index_t *index = index_load(file, filename);
    iterator_t *iter = query(idx, header, region);
    record_t *record = record_init();

    while (iterator_next(file, iter, record) >= 0) {
        /* Do stuff with record */
    }

    iterator_destroy(iter);
    record_destroy(record);
    file_close(file);

    return 0;
}

I have written the following Go code:

func main() {
    file := Open(filename)
    record := NewRecord()
    iter := file.Query(region)
    for {
        n, err := file.Next(iter, record)
        if err != nil {
            log.Fatal(err)
        }
        if n <= 0 {
            // No more records to read.
            break
        }
    }
}

This works, in the sense that it will allow me to access the records in a particular query region.

My question is whether this is an idiomatic way to approach this task in Go, or are there better alternatives? I've seen sites such as http://ewencp.org/blog/golang-iterators, but seem unable to get these examples to work with the C library (I was thinking it may be because the C library is reusing the record_t variable on each iteration rather than creating a new variable, but maybe it's just my lack of experience with Go).

Opinions will differ, but yeah, I think this is fine--it's essentially how you move through an input stream with `io.Reader`, except that you check for `n <= 0` instead of `err == io.EOF`. [`buffer.Scan` uses another approach](https://golang.org/pkg/bufio/#example_Scanner_lines) that you might find interesting to compare to as well. — twotwotwo, Sep 25 '15 at 00:20
I read the blog post you link to when it came out and I'd only use the callback/closure/channels approaches if something specific to the situation made them useful (e.g., if you want multiple consumers on a channel). Also, doesn't apply to your situation, but Go idiom is less strict than, say, C++ or Java's about exposing your object's data, so you might just expose a slice or map member on a struct. — twotwotwo, Sep 25 '15 at 00:32
@twotwotwo thanks, I found the link to buffer.Scan very useful. — jjellis, Sep 26 '15 at 10:57
Since there's been no other activity here, I'll expand and answer-ify the comments. — twotwotwo, Sep 27 '15 at 18:12

twotwotwo · Accepted Answer · 2015-09-27T21:00:29.003

What you're doing isn't that different from moving through a file with an io.Reader:

err, n := error(nil), 0
for err == nil {
    err, n = f.Read(in)
    // ...do stuff with in[:n]...
}

Or using (*bufio.Scanner).Scan() (see the docs):

for scanner.Scan() {
    // ...do something with scanner.Text()...
}
if err := scanner.Err(); err != nil {
    log.Fatalln(err)
}

I think you rarely want the more exotic iterator options in the blog post you linked to, the ones with closures or channels. Channels in particular invoke a lot of machinery meant for coordinating real threaded workloads, and in terms of convention, it's just typical in Go for loops to look a little different depending on what they're iterating. (In that respect, it's like iteration in C, but different from (say) Python, C++, or Java.)

idiomatic way to use c style iterator with Go

1 Answers1