4

We want our Go application to listen to the data changes on a collection. So, googling in search for a solution, we came across MongoDB's Change Streams. That link also exhibits some implementation snippets for a bunch of languages such as Python, Java, Nodejs etc. Yet, there is no piece of code for Go.

We are using Mgo as a driver but could not find explicit statements on change streams.

Does anyone have any idea on how to watch on Change Streams using that Mgo or any other Mongo driver for Go?

icza
  • 389,944
  • 63
  • 907
  • 827
vahdet
  • 6,357
  • 9
  • 51
  • 106

2 Answers2

17

The popular mgo driver (github.com/go-mgo/mgo) developed by Gustavo Niemeyer has gone dark (unmaintained). And it has no support for change streams.

The community supported fork github.com/globalsign/mgo is in much better shape, and has already added support for change streams (see details here).

To watch changes of a collection, simply use the Collection.Watch() method which returns you a value of mgo.ChangeStream. Here's a simple example using it:

coll := ... // Obtain collection

pipeline := []bson.M{}

changeStream := coll.Watch(pipeline, mgo.ChangeStreamOptions{})
var changeDoc bson.M
for changeStream.Next(&changeDoc) {
    fmt.Printf("Change: %v\n", changeDoc)
}

if err := changeStream.Close(); err != nil {
    return err
}

Also note that there is an official MongoDB Go driver under development, it was announced here: Considering the Community Effects of Introducing an Official MongoDB Go Driver

It is currently in alpha (!!) phase, so take this into consideration. It is available here: github.com/mongodb/mongo-go-driver. It also already has support for change streams, similarly via the Collection.Watch() method (this is a different mongo.Collection type, it has nothing to do with mgo.Collection). It returns a mongo.Cursor which you may use like this:

var coll mongo.Collection = ... // Obtain collection

ctx := context.Background()

var pipeline interface{} // set up pipeline

cur, err := coll.Watch(ctx, pipeline)
if err != nil {
    // Handle err
    return
}
defer cur.Close(ctx)

for cur.Next(ctx) {
    elem := bson.NewDocument()
    if err := cur.Decode(elem); err != nil {
        log.Fatal(err)
    }

    // do something with elem....
}

if err := cur.Err(); err != nil {
    log.Fatal(err)
}
icza
  • 389,944
  • 63
  • 907
  • 827
  • 2
    Great answer, notable suggestion. Thanks! – vahdet Mar 07 '18 at 12:17
  • @icza: This might be a noob question but I came from Node.js, how to obtain collection? – 0xh8h Mar 19 '19 at 10:50
  • 1
    @piavgh It depends on the driver. In `mgo`, use [`Database.C()`](https://godoc.org/github.com/globalsign/mgo#Database.C). In `mongo-go`, use [`Database.Collection()`](https://godoc.org/go.mongodb.org/mongo-driver/mongo#Database.Collection). – icza Mar 19 '19 at 11:05
5

This example uses the The MongoDB supported driver for Go with stream pipeline (filtering only documents having field1=1 and field2=false):

    ctx := context.TODO()
    clientOptions := options.Client().ApplyURI(mongoURI)
    client, err := mongo.Connect(ctx, clientOptions)
    if err != nil {
        log.Fatal(err)
    }
    err = client.Ping(ctx, nil)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Connected!")

    collection := client.Database("test").Collection("test")

    pipeline := mongo.Pipeline{bson.D{
        {"$match",
            bson.D{
                {"fullDocument.field1", 1},
                {"fullDocument.field2", false},
            },
        },
    }}
    streamOptions := options.ChangeStream().SetFullDocument(options.UpdateLookup)

    stream, err := collection.Watch(ctx, pipeline, streamOptions)
    if err != nil {
        log.Fatal(err)
    }
    log.Print("waiting for changes")
    var changeDoc map[string]interface{}
    for stream.Next(ctx) {
        if e := stream.Decode(&changeDoc); e != nil {
            log.Printf("error decoding: %s", e)
        }
        log.Printf("change: %+v", changeDoc)
    }
Arsen
  • 118
  • 1
  • 6