Firebase infinite scroll proper solution?

Question

I have built a pretty complex list(s) in a component that all query the same way. When I hit the end of my list, I increase my fetch limit by 10... Everything was going great until I realized I am loading all the data over again (not just the new 10). I want to avoid all these reads to reduce costs - I am not sure if firebase ignores the data that is already loaded or not... My list is running quite smooth when I fetch the extra 10 every time but I don't know if behind the scenes things will come back to haunt me!

UPDATE

I have updated my code based off some helpful feedback and I believe I have things working how they should be. There is only one thing that I am concerned about. When I reach the bottom of my list after my first fetch I am passing all of the previous data to my action creator then concatinating it to the new fetched data. This repeats everytime I hit the bottom as the list grows. My list is going to have over 1000 records so I am worried about potential performance issues, should I be? Have a look at my new attempt below!

Original Attempt:

onEndReached = () => {
 const { searchFilterText, effectVal } = this.state;

   this.setState({
     strainFetchIndex: this.state.strainFetchIndex + 10
   }, () => {
     const offset = this.state.strainFetchIndex;

      if (searchFilterText === '') {
       this.props.strainsFetch(offset);
     } else if (searchFilterText === 'Hybrid' || searchFilterText === 'Salt' || searchFilterText === 'Initial') {
       this.props.filterStrainAction(searchFilterText, offset);
     } else if (searchFilterText === 'Effects') {
       this.props.filterByEffect(effectVal, offset);
     }
  });
}

//HERES 1 of 4 ACTION CREATORS WHERE I FETCH MORE DATA (ALL ARE SIMILAR)

    export const strainsFetch = (offset) => {
      const ting = offset || 1;
        return (dispatch) => {
          firebase.database().ref('/strains')
            .orderByKey()
            .limitToFirst(1 * ting)
            .on('value', snapshot => {
              dispatch({ type: STRAINS_FETCH_SUCCESS, payload: snapshot.val() });
            });
        };
      };

New attempt:

  onEndReached = () => {
    const { searchFilterText } = this.state;
    const { lastKey } = this.props;
    const currentStrains = this.props.strains;

      if (this.state.filterLabel === 'Favourites') {
        return null;
      }
      if (searchFilterText === '') {
        //here I pass all previous records along with the last key (last key comes from my action creator)
        this.props.strainsFetch(currentStrains, lastKey);
      } 
    }



    //ACTION CREATOR



    export const strainsFetch = (currentStrains, lastKey) => {
      if (!lastKey) {
        return (dispatch) => {
          // console.log('first Fetch');
          firebase.database().ref('/strains')
            .orderByKey()
            .limitToFirst(10)
            .on('value', snapshot => {
              const snap = snapshot.val();
              const snapKeys = Object.keys(snap);
              const createLastKey = snapKeys[9];

              dispatch({ type: STRAINS_FETCH_SUCCESS, payload: snapshot.val(), key: createLastKey });
            });
        };
      }
        return (dispatch) => {
          // console.log('subsequent Fetch');
          firebase.database().ref('/strains')
            .orderByKey()
            .startAt(`${lastKey}`)
            .limitToFirst(11)
            .on('value', snapshot => {
              const snap = snapshot.val();
              const snapKeys = Object.keys(snap)
              .slice(1);

              const results = snapKeys
                 .map((key) => snapshot.val()[key]);

              const createLastKey = snapKeys[snapKeys.length - 1];
              const concatStrains = _.concat(currentStrains, results);

              dispatch({ type: STRAINS_FETCH_SUCCESS, payload: concatStrains, key: createLastKey });
            });
        };
      };



      //HERE IS WHAT MY REDUCER LOOKS LIKE



    import {
      STRAINS_FETCH_SUCCESS
    } from '../actions/types';

    const INITIAL_STATE = {};

    export default (state = INITIAL_STATE, action) => {
      switch (action.type) {
        case STRAINS_FETCH_SUCCESS:
        // console.log(action.payload);
          return { strains: action.payload, lastKey: action.key };

        default:
          return state;
      }
    };

Is this a bad practice to repeatedly pass the previous data to my action creator over and over as the list continues to grow? Or is there a more efficient way to accomplish this?

Thanks everyone!

Cheers.

Answer 1

You're correct in that you are re-loading all of the old ones in addition to the new ones. If you only want to load the new ones, then you should use the .startAt() method, check out the relevant docs here.

Your final setup would look something like:

export const strainsFetch = (offset) => {
    const startingIndex = offset || 1;
    const numRecordsToLoad = 10;
    return (dispatch) => {
        firebase.database().ref('/strains')
            .orderByKey()
            .startAt(startingIndex)
            .limitToFirst(numRecordsToLoad)
            .on('value', snapshot => {
                dispatch({ type: STRAINS_FETCH_SUCCESS, payload: snapshot.val() });
            });
    };
};