Scylla gocqlx how to implement pagination similar to a cursor

Question

I'm using Scylla to save parties created by a users. The method below returns a list of parties created by a user. I currently return all parties without allowing pagination, but I'm trying to implement Pagination for the method below but I still don't quite understand how pagination is handled with Scylla.
My guess would be that a cursor can be passed to a query. Based on this example it looks like the PageState can be used to pass something similar to a cursor.
I would appreciate a short explanation what PageState is and if I should use it to accomplish token based pagination. It would also be great if an example could be provided that shows how a new PageState can be returned to the client and used to fetch a new page on a second request.

func (pq *partyQuery) GetByUser(ctx context.Context, uId string) ([]datastruct.Party, error) {
    var result []datastruct.Party
    stmt, names := qb.
        Select(TABLE_NAME).
        Where(qb.Eq("user_id")).
        ToCql()

    err := pq.sess.
        Query(stmt, names).
        BindMap((qb.M{"user_id": uId})).
        PageSize(10).
        Iter().
        Select(&result)
    if err != nil {
        log.Println(err)
        return []datastruct.Party{}, errors.New("no parties found")
    }

    return result, nil
}

Thanks in advance and I appreciate your time.

Edit

For anybody interested, this is how I transformed my function to allow paging:

func (pq *partyQuery) GetByUser(ctx context.Context, uId string, page []byte) (result []datastruct.Party, nextPage []byte, err error) {
    stmt, names := qb.
        Select(TABLE_NAME).
        Where(qb.Eq("user_id")).
        ToCql()

    q := pq.sess.
        Query(stmt, names).
        BindMap((qb.M{"user_id": uId}))
    defer q.Release()

    q.PageState(page)
    q.PageSize(10)

    iter := q.Iter()
    err = iter.Select(&result)
    if err != nil {
        log.Println(err)
        return []datastruct.Party{}, nil, errors.New("no parties found")
    }

    return result, iter.PageState(), nil
}

score 2 · Accepted Answer · answered Mar 07 '22 at 10:07

2

Hi gocqlx author here.

Please take a look at this example https://github.com/scylladb/gocqlx/blob/25d81de30ebcdfa02d3d849b518fc57b839e4399/example_test.go#L482

    getUserVideos := func(userID int, page []byte) (userVideos []Video, nextPage []byte, err error) {
        q := videoTable.SelectQuery(session).Bind(userID)
        defer q.Release()
        q.PageState(page)
        q.PageSize(itemsPerPage)
    
        iter := q.Iter()
        return userVideos, iter.PageState(), iter.Select(&userVideos)
    }

You need to send page state to caller.

answered Mar 07 '22 at 10:07

mmatczuk

539
4
9

Thanks for your help. I got it working to return 10 rows and the next page state in an array of bytes. I use `Echo` and return it as json, which transforms the array of bytes to a string. This is an example page state as a string: `AAAAAMYAA..........AAAAD/////AAAAAA==`. But when I now convert this string back to `[]byte` and pass it to the `PageState()` method I get this error: `Invalid value for the paging state`. How can I encode/decode the `pageState` to reuse it on a second reuqest? – JonasLevin Mar 07 '22 at 16:55
Full Page State as string: `AAAAAMYAAAAhAAAAAQAAABUAAAB5Q3ZXLW9acVByaVc4d2FkdWtGMWsBLQAAAAIAAAAIAAAAAAABf2UwGAYVAAAANGRleGUyemJsM0dpREFtbE1QZEFS9f///xpJSs1RBGleSQ5ZFtoc84ABAAAAOQAAAAEZAAAAFAAAAAEAAAAIAAAAQkn+E9Uix7QBARkAAAAUAAAAAQAAAAgAAABCSf4T1SLHtAEBAQAAAEpCWQzs3IRCEdQBz/jrj64BAAAAAAD/////AAAAAA==` – JonasLevin Mar 07 '22 at 16:55
1

You must encode and decode the token. This looks like base64 it needs to be decoded. – mmatczuk Mar 08 '22 at 18:06

Scylla gocqlx how to implement pagination similar to a cursor

Edit

1 Answers1