How to access MongoDB Change Stream data using Go

Question

I am watching a mongodb collection with for all insert events using the golang. I used the mongodb function, changes streams.

My requirements is to access the data inside that return event where it seems to be of type bson.m

Here's my code sample:

func watch_for_events() {

    fmt.Println("Watching Started....")

    matchPipeline := bson.D{{"$match", bson.D{{"operationType", "insert"}}}}

    // open a change stream with an empty pipeline parameter
    coll := mongoConnection.Database(mongo_db).Collection(added_collection)
    changeStream, err := coll.Watch(context.TODO(), mongo.Pipeline{matchPipeline})
    if err != nil {
        panic(err)
    }
    defer changeStream.Close(context.TODO())
    // iterate over the cursor to print the change stream events
    for changeStream.Next(context.TODO()) {
        fmt.Println(changeStream.Current)
        fmt.Printf("POD name is: %v", changeStream.Current.Lookup("pod_name"))
    }

    fmt.Println("Watching Ended....")
}

This is the output I am getting

map[_id:map[_data:82637B3725000000012B022C0100296E5A10046C554EC75E644A81AF98CC28BAF03C0246645F69640064637B37253BC63551C4856EB50004] clusterTime:{1669019429 1} documentKey:map[_id:ObjectID("637b37253bc63551c4856eb5")] fullDocument:map[_id:ObjectID("637b37253bc63551c4856eb5") containers_and_images:[[nginx] [nginx:stable]] pod_name:nginx-847c4cd46c-dn2sc total_container_count:1] ns:map[coll:app1-added-pods db:ng-db] operationType:insert]

But the line fmt.Printf("POD name is: %v", changeStream.Current.Lookup("pod_name")) says no element found. I need to get the data such as, pod_name, namespace, total_container_count etc.

Can someone please help me with this?

icza · Answer 1 · 2022-11-21T14:22:30.410

The change event available in the ChangeStream.Current field is a document that has a fullDocument field holding the inserted full document. And note that changeStream.Current is of type bson.Raw, and Raw.Lookup() takes a path in the form of a variadic string slice:

func (r Raw) Lookup(key ...string) RawValue

So in order to get the pod_name field of the inserted document, do it like this:

fmt.Printf("POD name is: %v",
    changeStream.Current.Lookup("fullDocument", "pod_name"))

Also note that if you want other information from the change event or the inserted document, you may decode the change event into a Go value using ChangeStream.Decode().

Using a simple map, this is how you can access pod_name:

for changeStream.Next(context.TODO()) {
    var event struct {
        Doc bson.M `bson:"fullDocument"`
    }
    if err := changeStream.Decode(&event); err != nil {
        fmt.Printf("Failed to decode event: %v", err)
        continue
    }
    fmt.Printf("POD name is: %v", event.Doc["pod_name"])
}

If you already have a type modeling your documents, of course you can use that. Let's assume this is your type modeling the documents:

type YourDocument struct {
    ID      primitive.ObjectID `bson:"_id"`
    PodName string             `bson:"pod_name"`
    // Other properties
}

Using that:

for changeStream.Next(context.TODO()) {
    var event struct {
        Doc YourDocument `bson:"fullDocument"`
    }
    if err := changeStream.Decode(&event); err != nil {
        fmt.Printf("Failed to decode event: %v", err)
        continue
    }
    fmt.Printf("POD name is: %v", event.Doc.PodName)
}

How to access MongoDB Change Stream data using Go

1 Answers1