There are a couple of ways you can go about this problem:
Rendering all results with infinite loading
onAllData will give the number of results specified in the size prop. Usually, its not a good idea to set a very high size since it will take more time fetching and rendering the results. A good alternative for this is to use infinite scrolling by setting the pagination prop to false and setting a value in size prop which tells the component how many results to fetch when you reach the end of the list.
Fetching all results using scroll API
Note
This answer uses reactivesearch for web (for demonstration) but you can use the same props in reactivesearch-native since the API is same.
The above approach works fine if you're only interested in rendering the results. But if you wish to fetch all the results for the current query, you can use the scroll API from Elasticsearch. You can use ReactiveList to get the current query and then use it together with the scroll API.
For this purpose, you can make use of the onQueryChange prop on ReactiveList:
First specify an onQueryChange prop on the ReactiveList which receives the previous and current query as parameters:
onQueryChange={(prev, next) => ...}
This function will be invoked each time the query is changed so you can write a logic to fetch the hits for the current query (received in the next parameter) whenever required.
A scroll API call returns the results in the following format:
{
"_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAgAAAAClGlY4FlotbmJJZXA0U09lMlZFMUNyQ3M2M0EAAAAApRpWORZaLW5iSWVwNFNPZTJWRTFDckNzNjNB",
"took": 0,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 9407,
"max_score": 1,
"hits": [
{
"_index": "good-books-ds",
"_type": "good-books-ds",
"_id": "5676",
"_score": 1,
"_source": {
"authors": "Arthur C. Clarke, Gentry Lee",
"average_rating": 3.76,
"average_rating_rounded": 4,
"books_count": 48,
"id": 5676,
"image": "https://images.gr-assets.com/books/1375814957l/112518.jpg",
"image_medium": "https://images.gr-assets.com/books/1375814957m/112518.jpg",
"isbn": "1857230213",
"language_code": "eng",
"original_publication_year": 1991,
"original_series": "Rama",
"original_title": "The Garden of Rama (Rama, #3)",
"ratings_count": 16389,
"title": "The Garden of Rama (Rama, #3)"
}
},
{
"_index": "good-books-ds",
"_type": "good-books-ds",
"_id": "5681",
"_score": 1,
"_source": {
"authors": "Darren Shan",
"average_rating": 4.22,
"average_rating_rounded": 4,
"books_count": 52,
"id": 5681,
"image": "https://s.gr-assets.com/assets/nophoto/book/111x148-bcc042a9c91a29c1d680899eff700a03.png",
"image_medium": "https://s.gr-assets.com/assets/nophoto/book/111x148-bcc042a9c91a29c1d680899eff700a03.png",
"isbn": "",
"language_code": "",
"original_publication_year": 2003,
"original_series": "Cirque Du Freak",
"original_title": "Killers of the Dawn (Cirque Du Freak, #9)",
"ratings_count": 18194,
"title": "Killers of the Dawn (Cirque Du Freak, #9)"
}
},
{
"_index": "good-books-ds",
"_type": "good-books-ds",
"_id": "5683",
"_score": 1,
"_source": {
"authors": "Laura Joffe Numeroff, Felicia Bond",
"average_rating": 4.16,
"average_rating_rounded": 4,
"books_count": 13,
"id": 5683,
"image": "https://s.gr-assets.com/assets/nophoto/book/111x148-bcc042a9c91a29c1d680899eff700a03.png",
"image_medium": "https://s.gr-assets.com/assets/nophoto/book/111x148-bcc042a9c91a29c1d680899eff700a03.png",
"isbn": "60278684",
"language_code": "",
"original_publication_year": 2000,
"original_series": "",
"original_title": "If You Take a Mouse to the Movies",
"ratings_count": 17938,
"title": "If You Take a Mouse to the Movies"
}
},
{
"_index": "good-books-ds",
"_type": "good-books-ds",
"_id": "5685",
"_score": 1,
"_source": {
"authors": "Orson Scott Card, James Cameron",
"average_rating": 4.06,
"average_rating_rounded": 4,
"books_count": 15,
"id": 5685,
"image": "https://images.gr-assets.com/books/1225165505l/40289.jpg",
"image_medium": "https://images.gr-assets.com/books/1225165505m/40289.jpg",
"isbn": "99690608",
"language_code": "eng",
"original_publication_year": 1989,
"original_series": "",
"original_title": "The Abyss",
"ratings_count": 16318,
"title": "The Abyss"
}
},
{
"_index": "good-books-ds",
"_type": "good-books-ds",
"_id": "5687",
"_score": 1,
"_source": {
"authors": "Katarina Bivald, Alice Menzies",
"average_rating": 3.56,
"average_rating_rounded": 4,
"books_count": 63,
"id": 5687,
"image": "https://images.gr-assets.com/books/1452107441l/25573977.jpg",
"image_medium": "https://images.gr-assets.com/books/1452107441m/25573977.jpg",
"isbn": "149262344X",
"language_code": "eng",
"original_publication_year": 2013,
"original_series": "",
"original_title": "Läsarna i Broken Wheel rekommenderar",
"ratings_count": 14571,
"title": "The Readers of Broken Wheel Recommend"
}
}
]
}
}
The value received as _scroll_id can be passed to the scroll API to fetch the next set of results and so on till the number of hits is zero.
Note
If your cluster has a lot of data, its not a good idea to run this logic to fetch all the results every time the query changes. You can add a condition to limit the number of results fetched or store the current query in state and only fetch all the results when required.
Here's an example on how you can implement this with ReactiveList. In the example I'm fetching the results each time the query changes but you can modify it to fetch results conditionally instead:
In your render function:
<ReactiveList
...
size={10}
onQueryChange={this.handleQueryChange}
/>
Here's how the handleQueryChange function can look like. This will give you all the results for the current query:
handleQueryChange = async (prev, next) => {
// avoid fetching the results for match_all query since dataset is large
if (next && !next.query.match_all) {
console.log('Fetching all results for query:', next);
// modify the query size here if needed (currently it is 10)
// initial url to obtain scroll id is different
const initialResults = await this.fetchResults(next, url);
// keep scrolling till hits are present
// NOTE: careful if you've a lot of results,
// in that case you might want to add a condition to limit calls to scroll API
const scrollResults = await this.fetchScrollResults({
scroll: "1m",
scroll_id: initialResults._scroll_id
});
// combine the two to get all results
// concat hits from initialResults with hits from scrollResults
const allResults = initialResults.hits.hits.concat(scrollResults);
console.log(`${allResults.length} results found:`, allResults);
}
};
Its using two functions for fetching the results initially and later with the scroll_id. The endpoints for both are different, which you can find in the demo. Here's how the first fetchResults looks like:
fetchResults = (query, api) => {
return fetch(api, {
method: "POST",
headers: {
"content-type": "application/json",
Authorization: `Basic ${btoa(credentials)}`
},
body: JSON.stringify(query)
})
.then(res => res.json())
.catch(err => console.error(err));
};
The fetchScrollResults will use the scroll API to fetch results till the hits obtained are 0.
fetchScrollResults = async query => {
const res = await this.fetchResults(query, scrollUrl);
const { hits } = res.hits;
if (hits.length) {
return [
...hits,
...(await this.fetchScrollResults({
scroll: "1m",
scroll_id: res._scroll_id
}))
];
}
return [];
};
Check the demo, results will appear in the console.
