How do you manage effective data fetching with nested cursors in a relay-esk schema (with a SQL data source)?
- Do you try make a single complicated SQL query to resolve the N+1 problem with a "LIMIT args_first", "ORDER BY args_orderby" and "WHERE cursor > :args_after"
- Do you run 2 queries to the DB and make use of facebook data loader?
For example, I've got a schema structured as below:
enum BookSortKeys {
ID,
TITLE,
PRICE,
UPDATED_AT,
CREATED_AT
}
enum ReviewSortKeys {
ID,
REVIEW,
UPDATED_AT,
CREATED_AT
}
type Book {
id: ID!
title: String!
description: String
price: Float!
updatedAt: String!
createdAt: String!
reviews("""
Returns the elements that come after the specified cursor.
"""
after: String
"""
Returns the elements that come before the specified cursor.
"""
before: String
"""
Returns up to the first `n` elements from the list.
"""
first: Int
"""
Returns up to the last `n` elements from the list.
"""
last: Int
"""
Reverse the order of the underlying list.
"""
reverse: Boolean = false
"""
Sort the underlying list by the given key.
"""
sortKey: ReviewSortKeys = ID): ReviewConnection!
}
type Query {
books("""
Returns the elements that come after the specified cursor.
"""
after: String
"""
Returns the elements that come before the specified cursor.
"""
before: String
"""
Returns up to the first `n` elements from the list.
"""
first: Int
"""
Returns up to the last `n` elements from the list.
"""
last: Int
"""
Supported filter parameters:
- `title`
- `id`
- `price`
- `description`
- `created_at`
- `updated_at`
"""
query: String
"""
Reverse the order of the underlying list.
"""
reverse: Boolean = false
"""
Sort the underlying list by the given key.
"""
sortKey: BookSortKeys = ID): BookConnection!
}
type ReviewConnection {
pageInfo: PageInfo!
edges: [ReviewEdge!]!
}
type ReviewEdge {
cursor: String!
node: Review!
}
type BookConnection {
pageInfo: PageInfo!
edges: [BookEdge!]!
}
type BookEdge {
cursor: String!
node: Book!
}
type PageInfo {
hasNextPage: Boolean!
hasPreviousPage: Boolean!
}
type Review {
review: String!
id: ID!
updatedAt: String!
createdAt: String!
}
type Mutation {
}
schema {
query: Query
mutation: Mutation
}
And I'd like to execute a query like the below and retrieve the data in the most efficient manner.
query GET_BOOKS {
books(first:10, sortKey: PRICE, reverse: true) {
pageInfo {
hasNextPage
hasPreviousPage
}
edges {
cursor
node {
id
title
description
reviews(after:"base64-cursor" first: 5, sortKey: CREATED_AT) {
edges {
node{
review
}
}
}
}
}
}
}
I can very easily convert all of the pagination parameters for the top query (book) into a sql statement but with the nested cursor, I can only see 2 options (mentioned above)... current issues I'm facing before implementing these options are:
- If I go the pure SQL approach - is there even a clean way to run a single query and apply the
LIMIT
andWHERE createdAt > :after_cursor_val
at the nested (JOIN) level - If the above is possible, is it more performant than dataloader at scale? As the query seems like it will be pretty verbose and complex if implemented.
- What happens if the nested pagination tree grows (i.e. requests with 4 nested paginations)? Would a pure Query object level sql command suffice here? or is it more scalable to add resolvers on each relationship (i.e. book -> reviews has a sql query to pull all this book's specific reviews, reviews -> publications has a query to pull all the review's specific publications it's been in and so on and batch them in data loader)
- if you go the dataloader route, the batching seems to use a "WHERE IN" clause, (i.e.
SELECT * FROM reviews "reviews" WHERE "reviews".bookId IN (...list of book ids batched)
- would addingLIMIT
,ORDER BY
andWHERE createdAt > :cursor
provide unexpected results as my result set is a mix of entries across multiple 'book ids'? - long term, my personal feeling is that the pure sql approach is going to be messy from a code perspective, thoughts on this?