RxJS Polling for row updates on infinite scroll

Question

I was watching Matthew Podwysocki event on https://www.youtube.com/watch?v=zlERo_JMGCw 29:38

Where he explains how they solved scroll on netflix. Where user scroll for more data as previous data gets cleaned up and more adds up (but scroll back shows previous data again).

I wanted to do similar, but I grabbed netflix demo code:

function getRowUpdates(row) {
  var scrolls = Rx.Observable.fromEvent(document, 'scroll');
  var rowVisibilities = 
    scrolls.throttle(50)
      .map(function(scrollEvent) {
        return row.isVisible(scrollEvent.offset);
      })
      .distinctUntilChanged();

  var rowShows = rowrowVisibilities.filter(function(v) {
    return v;
  });
  var rowHides = rowrowVisibilities.filter(function(v) {
    return !v;
  });

  return rowShows
    .flatMap(Rx.Observable.interval(10))
    .flatMap(function() {
      return row.getRowData().takeUntil(rowHides);
    })
    .toArray();
}

But I'm bit confused on how to pass new data or page data according to the scroll here.. Can someone give little explanation on how I can do the following:

• fetch first list (I can do that)
• fetch more list as user scroll down (using paging next page)
• remove previous fetched data from memory, and refetch on request (scroll up).

Answer 1

Here is how I would do it in general lines. If this seems to give you satisfaction, I'll edit and add more details.

Version 2 : (For first one, see edit changes)

Premises :

1. The tag containing the dynamic list will be called "the zone".
2. Each row of the list will be contained in another DIV which can contains anything.
3. A page is enough rows to cover the zone.
4. Three javascript "constants" : numberOfLinesOnFirstLoad, numberOfLinesOnPageLoad, numberOfLinesToLoadAfter
5. JavaScript variables to hold required data : rows[page#], heights[page#], currentPageNumber = 1, maxPageNumber = 0
6. page# : # is the page number
7. rows[page#] shall contains a way to get them back from database, not real DOM objects.

Steps / Events :

1. Add the zone tag.
2. Load numberOfLinesOnFirstLoad rows.
3. If total rows height inferior zone height multiplied by three, then load numberOfLinesToLoadAfter rows. Repeat step 3 if rows added, otherwise continue to step 4.
4. maxPageNumber +=1. Find the next rows that full fills the zone. Add those rows to rows["page" + maxPageNumber] (as an array). Calculate the height of those and add it in heights["page" + maxPageNumber].
5. Repeat step 4 until no more rows and then continue to step 6.
6. When scrolling down and page1 (which means last element of rows["page1"]) is not visible, add another below : page4.
7. maxPageNumber += 1. Load numberOfLinesOnPageLoad rows.
8. If total new rows height inferior zone height, then numberOfLinesToLoadAfter rows. Repeat step 8 if rows added, otherwise put total new rows height in heights["page" + maxPageNumber] and the new rows as array in rows["page" + maxPageNumber] and continue to the step after entering this one (so either 9 or 11).
9. Still scrolling down, if page2 is not visible, remove the page1 elements from DOM and adjust scroll position by removing page1.height (heights["page1"]).
10. Load page 5 (step 7 and 8).
11. So now, there is page2 to page5 in the zone which page2 and page5 are not visible. If page3 is fully visible, than page4 is not, otherwise a part of page3 and page4 are visible. (Just to indicate possibilities, but not important)
12. When scrolling up and page2 is starting to be visible (so last element of rows["page2"]), load page1 by using rows["page1"], add page1.height (heights["page1"]) to scroll position and remove page5 from DOM. Here you can either remove it from variables rows && heights and maxPageNumber -= 1, but you can also keep them so that reloading this page is done in one process (so loading a page would imply to check if page definition already exists in those variables).