3

I'm at a bit of an impasse here with a query that SHOULD by all accounts, return results. If I run the following query in mongosh:

    db.events.aggregate([
                 {$unwind:"$features"},
                 {$match:{
                    $and: [
                       {"features.geometry":{
                          $geoWithin:{
                             $centerSphere:[
                                [-125.0364331, 78.925942],
                                10/3963.2]
                          }
                       }},
                       {"features.properties.new_time":{$gte:ISODate("2020-04-05T05:00:00Z")}},
                       {"features.properties.new_time":{$lte:ISODate("2020-04-07T11:00:00Z")}}
                    ]
                 }}])

I get thousands of results that I expect. However, if I run this same query with the Golang MongoDB driver as such:

unwindStage := bson.D{{"$unwind","$features"}}
matchStage = bson.D{ 
                    {"$match",bson.D{                       
                       {"$and", bson.A{                             
                             bson.D{{"features.geometry",bson.D{                    
                                {"$geoWithin",bson.D{                                       
                                   {"$centerSphere",bson.A{selectorGeoPoint,radius}},               
                                }},                                                         
                             }}},                                                   
                             bson.D{{"properties.new_time", bson.D{{"$gte",startDateTime}, {"$lte",endDateTime}} }},
                       }},                                          
                    }},                                     
                 }
filterCursor, err := collection.Aggregate(ctx, mongo.Pipeline{unwindStage,matchStage}

fmt.Printf("CURSOR: %s\n", filterCursor)
fmt.Println("CURSOR SIZE: ", filterCursor.RemainingBatchLength())

I get a cursor returned with 0 length. The only thing I can figure is that the query I've constructed in Go somehow does not map identically to the one I've issued in mongosh? The only other thing I can think of, is that this query takes a long time (~15 sec even with indexes and I've had to increase the context timeout to about 30 seconds) and somehow that is impacting things?

For what it's worth, printing the cursor result shows the following:

"CURSOR: &{ %!s(*driver.BatchCursor=&{0xc000157520 0xc0000a3640 local events 0 <nil> 0xc00002e2a0 <nil> <nil> 0 0 0xc000020cc0 true <nil> [] <nil> <nil> false 0 0}) %!s(*bsoncore.DocumentSequence=<nil>) %!s(int=0) %!s(*bsoncodec.Registry=&{map[0x972220:0xc0000ac1ca 0x974ae0:0xc0000ac1c5 0x978000:0x5db000 0x97b080:0x5da6a0 0x97b0c0:0x5da740 0x97b540:0x5daf60 0x97bec0:0xc0000ac1cb 0x97fa40:0x5db0a0 0x993f00:0xc0000ac1ca 0x99c7a0:0xc0000ac1c6 0x9a6080:0xc0000ac1ca 0x9aae80:0x5da920 0x9aaee0:0x5dace0 0x9aaf40:0x5dac40 0x9aafa0:0x5da9c0 0x9ab000:0x5da880 0x9b5440:0x5da560 0x9d7a60:0x5dae20 0x9dcfe0:0xe9bc70 0x9e1580:0x5da7e0 0x9e1640:0x5daba0 0x9e8840:0x5da420 0x9ea880:0x5dab00 0x9ea940:0x5daa60 0x9f0860:0x5eb1c0 0x9f5480:0x5dad80 0x9fef00:0x5da4c0 0xa0d8e0:0x5da600 0xa27ea0:0xc0000ac1cc 0xa2b3c0:0x5eb120] map[0x974ae0:0xc0000ac1c5 0x97b080:{0x5d9220 0x5d92e0} 0x97b0c0:{0x5d93c0 0x5d9480} 0x99c7a0:0xc0000ac1c6 0x9a6080:0x5d82c0 0x9aae80:{0x5d86c0 0x5d8780} 0x9aaee0:{0x5d9080 0x5d9140} 0x9aaf40:{0x5d8ee0 0x5d8fa0} 0x9aafa0:{0x5d8860 0x5d8920} 0x9ab000:{0x5d8520 0x5d85e0} 0x9b5440:{0x5d98a0 0x5d9960} 0x9d7a60:{0x5d9ca0 0x5d9d60} 0x9dcfe0:0xe9bc70 0x9e1580:{0x5d8380 0x5d8440} 0x9e1640:{0x5d8d40 0x5d8e00} 0x9e8840:{0x5d9560 0x5d9620} 0x9ea880:{0x5d8ba0 0x5d8c60} 0x9ea940:{0x5d8a00 0x5d8ac0} 0x9f0860:0x5eb320 0x9f5480:0x5d9be0 0x9fef00:{0x5d9700 0x5d97c0} 0xa0d8e0:{0x5d9a40 0x5d9b00} 0xa27ea0:0xc0000ac1cc 0xa2b3c0:0x5eb260] [{0x9b2c00 0x5db140} {0x9b2900 0x5db1e0} {0x9b2980 0x5db280}] [{0x9b2c80 0x5da0a0} {0x9b2a80 0x5da160}] map[1:0x5daec0 2:0x5daf60 3:0x5daf60 4:0x5daf60 5:0x5daf60 6:0x5daf60 7:0xc0000ac1e8 8:0xc0000ac1e8 9:0xc0000ac1e8 10:0xc0000ac1e8 11:0xc0000ac1e8 13:0x5db000 14:0x5db000 17:0x5db0a0 21:0xc0000ac1c7 22:0xc000094b70 23:0xc0000ac1ca 24:0xc0000ac1cb 25:0xc0000be200] map[1:{0x5d9e40 0x5d9f00} 2:{0x5d7f80 0x5d8040} 3:{0x5d7f80 0x5d8040} 4:{0x5d7f80 0x5d8040} 5:{0x5d7f80 0x5d8040} 6:{0x5d7f80 0x5d8040} 7:0xc0000ac1e8 8:0xc0000ac1e8 9:0xc0000ac1e8 10:0xc0000ac1e8 11:0xc0000ac1e8 13:{0x5d8120 0x5d81e0} 14:{0x5d8120 0x5d81e0} 17:0x5d9fe0 21:0xc0000ac1c7 22:0xc000094c60 23:0xc0000ac1ca 24:0xc0000ac1cb 25:0xc0000be280] map[0:0x9a6080 1:0x978000 2:0x97bec0 3:0x9a6080 4:0x993f00 5:0x9e1580 6:0x9ab000 7:0x9e8840 8:0x977640 9:0x9aae80 11:0x9ea940 12:0x9ea880 13:0x97b080 14:0x97b0c0 15:0x9d7a60 16:0x97b5c0 17:0x9e1640 18:0x97b600 19:0x9fef00 127:0x9aaee0 255:0x9aaf40] {{0 0} 0 0 0 0}}) %!s(*session.Client=&{0xc0000bf0c0 [220 63 235 104 64 97 75 30 151 140 9 51 122 202 42 14] [] true <nil> 1 true false false false false false false <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> 0xc000168ba0 0 <nil> [] <nil> <nil>}) <nil>}"

What can I try next?

halfer
  • 19,824
  • 17
  • 99
  • 186
Jay
  • 75
  • 1
  • 6
  • Anyone have any idea? I'm still cracking my head on this. The only other thing I've discovered is that this doesn't seem to be a problem with smaller returns. In other words, if I have a return of say 10-20 documents I get a valid cursor. If I get into the hundreds of documents returned, then the cursor comes back as 0. I'm at a complete loss as to what the problem is. All I can possibly think of is that I'm bumping up against some limit. I've tried returning cursor.All, as well. No luck there either. – Jay Nov 15 '21 at 19:35
  • Did you check context? It looks like context canceled transaction. Try to write context.TODO() instead ctx in collection.Aggregate, to exclude it influence on db request – Eugene.S Nov 15 '21 at 20:06
  • That didn't change the behavior either unfortunately. I've tried several different versions. My context is declared as such: // DECLARE A CONTEXT OBJECT //ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) //ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second) ctx, cancel := context.WithTimeout(context.TODO(), 120*time.Second) – Jay Nov 15 '21 at 20:16
  • I've tried explicitly setting a batch size with opts := options.Aggregate().SetBatchSize(5000) This makes no difference either – Jay Nov 15 '21 at 20:26
  • No idea, but you can try to remove parts of the query from both sides until it produces the same results. Then slowly add parts of the query back in and see where things go wrong. – Fabian Nov 15 '21 at 23:37
  • So I stumbled across what is causing this but I don't understand why it is causing it. If I remove the "AND" in the match stage, and use only one of the conditions, the query works. Having done that, I tried re-creating the same query above except with 2 separate match stages (My understanding is mongo creates the AND from this anyway). This ALSO does not return results. Does anyone know if the Golang Mongo driver has an issue with AND'ing conditions in an aggregate pipeline?? – Jay Nov 16 '21 at 14:18
  • Dunno either just an idea, but can you remove AND and have it used implicitly? There are only two mentions of $and in the entire client codebase and dunno whether its supported very well (search site:https://github.com/mongodb/mongo-go-driver "$and" with google). Otherwise maybe ask on the support forum for the driver. – Fabian Nov 16 '21 at 15:36

0 Answers0