Limit size of diffable data source snapshots after calling performFetch

Question

Problem

After executing an NSBatchBatchInsertRequest and calling fetchedResultsController.performFetch(), the frc’s delegate method controller(_:didChangeContentWith:) is called with a NSDiffableDataSourceSnapshot<Section, NSManagedObjectID> that unexpectedly contains all data in Core Data (even though I've only just inserted new data).

Code

Here's how the NSFetchedResultsController is created:

    let fetchedResultsController = NSFetchedResultsController<MyManagedObject>(
        fetchRequest: fetchRequest,
        managedObjectContext: persistentContainer.viewContext,
        sectionNameKeyPath: nil,
        cacheName: nil
    )

The NSFetchRequest:

    let fetchRequest = MyManagedObject.fetchRequest()
    fetchRequest.sortDescriptors = [NSSortDescriptor(key: "value", ascending: true)]
    fetchRequest.fetchBatchSize = 10

The snapshot is applied in this NSFetchedResultsControllerDelegate method below.

func controller(_ controller: NSFetchedResultsController<NSFetchRequestResult>, didChangeContentWith snapshot: NSDiffableDataSourceSnapshotReference) {
    dataSource.apply(snapshot as NSDiffableDataSourceSnapshot<Section, NSManagedObjectID>, animatingDifferences: false)
}

Details

• The sample app receives 10,000 Item instances every 5 seconds and runs an NSBatchInsertRequest on a new background context with performAndWait to convert and store the Items as MyManagedObject instances.
• performFetch() is then called on main.
• Calling performFetch() ensures a new snapshot is provided in controller(_:didChangeContentWith:), which the data source can apply(_:animatingDifferences:completion:) to update the UI.
• The items populate a UICollectionView, which is hooked up to a UICollectionViewDiffableDataSource<Section, NSManagedObjectID>.
  • When the data source is asked to make a cell for a specific NSManagedObjectID, the right MyManagedObject for that NSManagedObjectID using fetchedResultsController.managedObjectContext.object(with:).
  • Then the MyManagedObject is converted into an Item, which is finally used to configure the cell.

Attempt at a solution

The most obvious solution to me was implementing this delegate method only, but that method doesn't seem to get called:

func controller(_ controller: NSFetchedResultsController<NSFetchRequestResult>, didChangeContentWith diff: CollectionDifference<NSManagedObjectID>) {
    print(diff)
}

I don't want the UI to stutter and hang when I'm fetching the data. I think that the reason that's happening is because my snapshots are huge. My guess why they're huge is because my fetch request fetches all the data (but I only want to add new data to the snapshot). To fix that, I've tried to setting fetchLimit to a small number like 2, but that just causes only 2 items to be fetched at all. I think I could workaround that by updating fetchOffset, but that approach could get messy as I scroll around. Adjusting fetchBatchSize doesn't make a difference. Memory is not leaking either. And FWIW, I'm applying the snapshots in a background queue, but that also doesn't seem to improve the UI stuttering.

I want to store items to Core Data as I get them, and then only fetch from the persistent store when scrolling to off-screen cells. That appears to be happening because I can see the data source only requests cells for index paths near the visible range when the snapshot is applied (even though the vertical scroll bar of the collection view is shrinking over time). But the size of snapshots (= the total number of items received until now) is ever increasing.

I can get the current snapshot and only append the items missing from the new snapshot, but that will still require me to go through a snapshot of unbounded size. How can I avoid doing that and load lots of data without the UI stuttering? I'm only inserting new data into Core Data, so why am I getting all the data in the snapshots?