1

I'm writing a Flutter application which uses the user's location to identify items around them. As it's only possible to do comparison queries against one field in Firebase, I'm using geohashes as per Frank van Puffelen's lecture on the subject.

I've got the initial query working fine:

qs = await Firestore.instance
        .collection('users')
        .where('geohash', isGreaterThanOrEqualTo: boundaryGeoHash1)
        .where('geohash', isLessThanOrEqualTo: boundaryGeoHash2)
        .getDocuments();

this returns all the people in the required geographic area, as expected. The problem is, I expect there to be a high volume of users, so I want to limit the amount of downloads per query to a certain number - say 10.

When I write this query (which I believed should do the trick), what actually seems to happen is that the table is ordered by 'geohash', it then queries the first 10 items in the table, and then sends me the records in the first 10 items that fulfil the requirements of the .where part of the query:

qs = await Firestore.instance
          .collection('users')
          .orderBy('geohash')
          .where('geohash', isGreaterThanOrEqualTo: boundaryGeoHash1)
          .where('geohash', isLessThanOrEqualTo: boundaryGeoHash2)
          .startAfterDocument(dS)
          .limit(10)
          .getDocuments();

//dS = last document from previous query

whereas what I want the query to do is to always send me 10 results, all of which fulfil the .where part of the query.

Using the test database I've generated, calling the initial query returns 114 results. The second one finds 2 which brings me to my conclusion as to what is happening.

Can what I'm trying to do be achieved? And if so, any clues as to what I'm doing wrong?

EDIT

As per Frank's request, dS is populated as follows:

if (qs != null) {
     dS = qs.documents[qs.documents.length - 1];
}

//dS is a global variable

EDIT 2

Here is the function which does the query on the database.

Future<List<CutDownUserProfile>> getReducedUserDataBetweenGeohashes(
      String boundaryGeoHash1, String boundaryGeoHash2, DocumentSnapshot dS, int downloadLimit) async {
    List<CutDownUserProfile> retn = List<CutDownUserProfile>();

    QuerySnapshot qs;
    if (dS != null) {
      print('ds != null');
      print('dS.userNumber = ' + dS.data['userNumber'].toString());
      print('dS.userID = ' + dS.data['userID']);
      print('dS.geohash = ' + dS.data['geohash']);
      print('dS.name = ' + dS.data['name']);

      qs = await Firestore.instance
          .collection('userHeaders')
          .orderBy('geohash')
          .where('geohash', isGreaterThanOrEqualTo: boundaryGeoHash1)
          .where('geohash', isLessThanOrEqualTo: boundaryGeoHash2)
          .startAfterDocument(dS)
          .limit(downloadLimit)
          .getDocuments();
    } else {
      print('ds == null');
      qs = await Firestore.instance
          .collection('userHeaders')
          .orderBy('geohash')
          .where('geohash', isGreaterThanOrEqualTo: boundaryGeoHash1)
          .where('geohash', isLessThanOrEqualTo: boundaryGeoHash2)
          .limit(downloadLimit)
          .getDocuments();
    }
    if (qs.documents.isNotEmpty) {
      print('through this! qs len = ' + qs.documents.length.toString());
      List<DocumentSnapshot> ds = qs.documents.toList();

      print('/////DS Len = ' + ds.length.toString());
      for (int i = 0; i < ds.length; i++) {
        CutDownUserProfile cDUP = CutDownUserProfile();

        cDUP.geoHash = ds[i].data['geohash'];
        Vector2 latLon = globals.decodeGeohash(cDUP.geoHash);

        double dist = getDistanceFromLatLonInKm(globals.localUser.homeLat, globals.localUser.homeLon, latLon.x, latLon.y);

        if (dist <= globals.localUser.maxDistance && dist <= ds[i].data['maxDist']) {
            CutDownUserProfile cDUP2 = CutDownUserProfile();

            cDUP2.userNumber = ds[i].data['userNumber'];
            cDUP2.userID = ds[i].data['userID'];
            cDUP2.geoHash = ds[i].data['geohash'];

            retn.add(cDUP2);
        }
      }
      if (qs != null) {
        globals.lastProfileSeen = qs.documents[qs.documents.length - 1];
      }
    } else {
      print('no results');
    }
}

The line print('/////DS Len = ' + ds.length.toString()); prints out the length of the queries returned from the search, and, as mentioned earlier returns 2. Without the .orderBy the .startAfterDocument(dS) and the .limit(downloadLimit) the code returns 114 results, which is what I expected. For completeness, here is the original code:

Future<List<CutDownUserProfile>> getReducedUserDataBetweenGeohashesALL(String boundaryGeoHash1, String boundaryGeoHash2) async {
    List<CutDownUserProfile> retn = List<CutDownUserProfile>();

    await Firestore.instance
        .collection('userHeaders')
        .where('geohash', isGreaterThanOrEqualTo: boundaryGeoHash1)
        .where('geohash', isLessThanOrEqualTo: boundaryGeoHash2)
        .getDocuments()
        .then((event) {
      if (event.documents.isNotEmpty) {
        List<DocumentSnapshot> ds = event.documents.toList(); //if it is a single document

        print('/////DS Len = ' + ds.length.toString());
        for (int i = 0; i < ds.length; i++) {
          CutDownUserProfile cDUP = CutDownUserProfile();

          cDUP.geoHash = ds[i].data['geohash'];
          Vector2 latLon = globals.decodeGeohash(cDUP.geoHash);

          double dist = getDistanceFromLatLonInKm(globals.localUser.homeLat, globals.localUser.homeLon, latLon.x, latLon.y);

          CutDownUserProfile cDUP2 = CutDownUserProfile();

          cDUP2.userNumber = ds[i].data['userNumber'];
          cDUP2.userID = ds[i].data['userID'];
          cDUP2.geoHash = ds[i].data['geohash'];

          retn.add(cDUP2);
        }
      } else {
        print('no results');
      }
    }).catchError((e) => print("error fetching data: $e"));
  • It sounds like your `dS` variable refers to a document that is close to the end of the range of matching documents. Can you edit your question to show how you populate `dS`? – Frank van Puffelen Jul 14 '20 at 13:19
  • Hey @FrankvanPuffelen! Done. The dS variable is actually a global (as queries are only done one at a time), and it's just set to the last item in the previously recovered query. That all works fine - the problem is it seems that the query is ordering the table first and THEN running the 'where' query on the first 10 items, as opposed to running the where query and returning the first 10 items - so in essence, unless the 10 records in the geohash field contain items which fulfil the 'where' query, I get nothing back, whereas what I want is the first 10 items which fulfil the 'where' query... – Anthony Rodgers Jul 14 '20 at 13:55
  • What you're describing is not how limits are supposed to work, nor how I've seen them work in order situations. Instead of describing your hypothesis, can you update your question to include a single snippet of code that reproduces the problem in a way that we can all look at, for example by printing the relevant fields in the output? – Frank van Puffelen Jul 14 '20 at 14:07
  • @FrankvanPuffelen added more info. :-) – Anthony Rodgers Jul 14 '20 at 14:24
  • Thanks for the update. It's getting a bit hard to follow though, and I still don't see the initial query (with a limit, but without a `startAfter`) that you use to determine the first `dS`. Is it possible to reproduce the problem in a single snippet that does nothing more than fire the two queries, and logging the offending data? This will typically mean that you start reproducing anew, instead of copy/pasting snippets from your existing code. It may be more work for you, but ensures we are both looking at the same minimal, complete/standalone repro. – Frank van Puffelen Jul 14 '20 at 14:54

0 Answers0