4

How do you manage effective data fetching with nested cursors in a relay-esk schema (with a SQL data source)?

  1. Do you try make a single complicated SQL query to resolve the N+1 problem with a "LIMIT args_first", "ORDER BY args_orderby" and "WHERE cursor > :args_after"
  2. Do you run 2 queries to the DB and make use of facebook data loader?

For example, I've got a schema structured as below:

enum BookSortKeys {
    ID,
    TITLE,
    PRICE,
    UPDATED_AT,
    CREATED_AT
}
enum ReviewSortKeys {
    ID,
    REVIEW,
    UPDATED_AT,
    CREATED_AT
}
type Book {
  id: ID!
  title: String!
  description: String
  price: Float!
  updatedAt: String!
  createdAt: String!
  reviews("""
    Returns the elements that come after the specified cursor.
    """
    after: String
    """
    Returns the elements that come before the specified cursor.
    """
    before: String
    """
    Returns up to the first `n` elements from the list.
    """
    first: Int
    """
    Returns up to the last `n` elements from the list.
    """
    last: Int
    """
    Reverse the order of the underlying list.
    """
    reverse: Boolean = false
    """
    Sort the underlying list by the given key.
    """
    sortKey: ReviewSortKeys = ID): ReviewConnection!
}
type Query {
  books("""
    Returns the elements that come after the specified cursor.
    """
    after: String
    """
    Returns the elements that come before the specified cursor.
    """
    before: String
    """
    Returns up to the first `n` elements from the list.
    """
    first: Int
    """
    Returns up to the last `n` elements from the list.
    """
    last: Int
    """
    Supported filter parameters:
     - `title`
     - `id`
     - `price`
     - `description`
     - `created_at`
     - `updated_at`
    """
    query: String
    """
    Reverse the order of the underlying list.
    """
    reverse: Boolean = false
    """
    Sort the underlying list by the given key.
    """
    sortKey: BookSortKeys = ID): BookConnection!
}
type ReviewConnection {
    pageInfo: PageInfo!
    edges: [ReviewEdge!]!
}
type ReviewEdge {
    cursor: String!
    node: Review!
}
type BookConnection {
    pageInfo: PageInfo!
    edges: [BookEdge!]!
}
type BookEdge {
    cursor: String!
    node: Book!
}
type PageInfo {
    hasNextPage: Boolean!
    hasPreviousPage: Boolean!
}
type Review {
    review: String!
    id: ID!
    updatedAt: String!
    createdAt: String!
}
type Mutation {
}
schema {
  query: Query
  mutation: Mutation
}

And I'd like to execute a query like the below and retrieve the data in the most efficient manner.

query GET_BOOKS {
  books(first:10, sortKey: PRICE, reverse: true) {
       pageInfo {
      hasNextPage
      hasPreviousPage
    } 
    edges {
      cursor
      node {
        id
        title
        description
        reviews(after:"base64-cursor" first: 5, sortKey: CREATED_AT) {
          edges {
            node{
              review
            }
          }
        }
      }
    }
  }
}

I can very easily convert all of the pagination parameters for the top query (book) into a sql statement but with the nested cursor, I can only see 2 options (mentioned above)... current issues I'm facing before implementing these options are:

  1. If I go the pure SQL approach - is there even a clean way to run a single query and apply the LIMIT and WHERE createdAt > :after_cursor_val at the nested (JOIN) level
  2. If the above is possible, is it more performant than dataloader at scale? As the query seems like it will be pretty verbose and complex if implemented.
  3. What happens if the nested pagination tree grows (i.e. requests with 4 nested paginations)? Would a pure Query object level sql command suffice here? or is it more scalable to add resolvers on each relationship (i.e. book -> reviews has a sql query to pull all this book's specific reviews, reviews -> publications has a query to pull all the review's specific publications it's been in and so on and batch them in data loader)
  4. if you go the dataloader route, the batching seems to use a "WHERE IN" clause, (i.e. SELECT * FROM reviews "reviews" WHERE "reviews".bookId IN (...list of book ids batched) - would adding LIMIT, ORDER BY and WHERE createdAt > :cursor provide unexpected results as my result set is a mix of entries across multiple 'book ids'?
  5. long term, my personal feeling is that the pure sql approach is going to be messy from a code perspective, thoughts on this?
Daniel Blignaut
  • 121
  • 1
  • 4
  • In the mean time, I've decided to take the data loader approach as a baseline implementation, assess performance from there and optimise specfic use cases. I think this coupled together with strong security measures is the best approach. [Such measures are discussed quite well here](https://blog.apollographql.com/securing-your-graphql-api-from-malicious-queries-16130a324a6b) for those interested. Let me know your thoughts! – Daniel Blignaut Jul 19 '19 at 12:13

0 Answers0