3

Below is the code that I'm using to implement pagination for data retrieved from the firebase realtime database. Basically, I'm trying to get the first n content according to page number, and then getting the last n content from the data retrieved in the first query.

function getUserSnapshotOrVerifyUserId(username, idToken, cb) {
    if (username == null || username.length == 0 || idToken == null || idToken.length == 0)
        return cb({
            status: "error",
            errorMessage: "Missing params."
        }, null);
    
    admin.auth().verifyIdToken(idToken).then(decodedToken => {
        let uid = decodedToken.uid;

        admin.database().ref().child("users").orderByChild("username").equalTo(username).once('value', snapshot => {
            if (!snapshot.exists())
                return cb({
                    status: "error",
                    message: "invalid-profile"
                });

            snapshot.forEach(child => {
                const id = child.val().id;
                if (id !== uid)
                    return cb({
                        status: "error",
                        message: "Invalid ID"
                    });

                admin.database().ref("users/" + id).once("value", snapshot => {
                    if (!snapshot.exists())
                        return cb({
                            status: "error",
                            errorMessage: "user not found."
                        });
                    
                    return cb(null, id, snapshot);
                });
            });
        });
    }).catch(err => cb({
        status: "error",
        message: err
    }));
}

exports.getUserContentTestPagination = functions.https.onRequest((req, res) => {
    corsHandler(req, res, async () => {
        try {
            const username = req.body.username || req.query.username;
            const idToken = req.body.idToken;

            const limit = 2;
            const page = req.body.page || 1;

            const limitToFirst = page * limit;
            const limitToLast = limit;


            getUserSnapshotOrVerifyUserId(username, idToken, async (err, id) => {
                if(err) return res.json(err);

                const uploadsRef = admin.database().ref('uploads').orderByChild('createdBy').equalTo(id)

                const firstnquery = uploadsRef.limitToFirst(limitToFirst);
                const lastnquery = firstnquery.limitToLast(limitToLast);

                lastnquery.once("value", snapshot => {
                    res.json({
                        snapshot
                    })
                })
                
            })
        } catch (err) {
            res.json({
                status: "error",
                message: err
            })
        }
    });
});

This is returning a function timeout, however, when I try to get the first n data using firstnquery, it is returning the first n data as expected. So the problem is with lastnquery. Any help would be appreciated.

UPDATE 1:

exports.getUserContentTestPagination = functions.https.onRequest((req, res) => {
    corsHandler(req, res, async () => {
        try {
            const username = req.body.username || req.query.username;
            const idToken = req.body.idToken;

            const limit = 2;
            const page = req.body.page || 1;
            
            let lastKnownKeyValue = null;

            getUserSnapshotOrVerifyUserId(username, idToken, async (err, id) => {
                if(err) return res.json(err);

                const uploadsRef = admin.database().ref('uploads');
                const pageQuery = uploadsRef.orderByChild('createdBy').equalTo(id).limitToFirst(limit);
                
                pageQuery.once('value', snapshot => {
                    snapshot.forEach(childSnapshot => {
                        lastKnownKeyValue = childSnapshot.key;
                    });

                    if(page === 1){
                        res.json({
                            childSnapshot
                        })
                    } else {
                        const nextQuery = uploadsRef.orderByChild('createdBy').equalTo(id).startAt(lastKnownKeyValue).limitToFirst(limit);

                        nextQuery.once("value", nextSnapshot => {
                            nextSnapshot.forEach(nextChildSnapshot => {
                                res.json({
                                    nextChildSnapshot
                                })
                            })
                        })
                    }

                });

            })
        } catch (err) {
            res.json({
                status: "error",
                message: err
            })
        }
    });
});
Cedric Hadjian
  • 849
  • 12
  • 25

1 Answers1

1

It is incredibly uncommon to use both limitToFirst and limitToLast in a query. In fact, I'm surprised that this doesn't raise an error:

const firstnquery = uploadsRef.limitToFirst(limitToFirst);
const lastnquery = firstnquery.limitToLast(limitToLast);

Firebase queries are based on cursors. This means that to get the data for the next page, you must know the last item on the previous page. This is different from most databases, which work based on offsets. Firebase doesn't support offset based queries, so you'll need to know the value of createdBy and the key of the last item of the previous page.

With that, you can get the next page of items with:

admin.database().ref('uploads')
     .orderByChild('createdBy')
     .startAt(idOfLastItemOfPreviousPage, keyOfLastItemOfPreviousPage)
     .limitToFist(pageSize + 1)

I highly recommend checking out some other questions on implementing pagination on the realtime database, as there are some good examples and explanations in there too.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • Yes I followed your answer on this link: https://stackoverflow.com/questions/35025419/firebase-results-range-using-startat-and-endat and came up with a solution I just updated my post with. However, I'm getting ` Starting point was already set (by another call to startAt or equalTo).` as obviously I'm using both `equalTo()` and `startAt()` at the same time. I can't not use equalTo as I have to filter out the content uploaded by a specific user using his `id` passed to `equalTo()`. I also ended up using "show more" solution rather pagination. Your advice would be appreciated, thank you. – Cedric Hadjian Feb 06 '21 at 16:25
  • You can't use both `startAt()` and `equalTo()` in a single query, which is why you see me pass two arguments to `.startAt(idOfLastItemOfPreviousPage, keyOfLastItemOfPreviousPage)` in my answer. The first is the value of `createdBy` to start at, the second is the key to start at in case there are multiple nodes with the same value for `createdBy`. – Frank van Puffelen Feb 06 '21 at 18:15
  • I still don't get this, if the uploads aren't sorted by the id of `createdBy`, how would it know I only need the uploads of that specific user? How would passing the value of `createdBy` to `startAt()` let it know that I need the uploads of that specific user? – Cedric Hadjian Feb 06 '21 at 18:29
  • The `idOfLastItemOfPreviousPage ` is the same value you used in `equalTo(id)` in your code (that's where I copied the name from), but then for the last item on the previous page. If that `id` value is a user ID (I recommend using better variable names in that case going forward), then it'd `userIdOfLastItemOfPreviousPage`. – Frank van Puffelen Feb 06 '21 at 18:51
  • I see, thank you. I'm trying to implement what you suggest now. 1 more question, how do I show the latest result to the user? I researched a bit and apparently, there's no way to order by desc the createdBy date. Maybe there's some sort of update now? And how would I tackle this issue when I'm implementing load more? – Cedric Hadjian Feb 06 '21 at 20:28
  • Okay, your help is appreciated. I think I can move forward, now that I sufficiently have the foundation of it. – Cedric Hadjian Feb 06 '21 at 21:28