Querying WordPress WP-API for lots of posts (using promises!)

Question

I am working on a Node.js application which uses the WordPress JSON API as a kind of headless CMS. When the application spins up, we query out to the WP database and pull in the information we need (using Axios), manipulate it, and store it temporarily.

Simple enough - but one of our post categories in the CMS has a rather large number of entries. For some godforsaken reason, WordPress has capped the API request limit to a maximum of 99 posts at a time, and requires that we write a loop that can send concurrent API requests until all the data has been pulled.

For instance, if we have 250 posts of some given type, I need to hit that route three separate times, specifying the specific "page" of data I want each time.

Per the docs, https://developer.wordpress.org/rest-api/using-the-rest-api/pagination/, I have access to a ?page= query string that I can use to send these requests concurrently. (i.e. ...&page=2)

I also have access to X-WP-Total in the headers object, which gives me the total number of posts within the given category.

However, these API calls are part of a nested promise chain, and the whole process needs to return a promise I can continue chaining off of.

The idea is to make it dynamic so it will always pull all of the data, and return it as one giant array of posts. Here's what I have, which is functional:

const request = require('axios');

module.exports = (request_url) => new Promise((resolve, reject) => {

  // START WITH SMALL ARBITRARY REQUEST TO GET TOTAL NUMBER OF POSTS FAST
  request.get(request_url + '&per_page=1').then(
    (apiData) => {

      // SETUP FOR PROMISE.ALL()
      let promiseArray = [];

      // COMPUTE HOW MANY REQUESTS WE NEED
      // ALWAYS ROUND TOTAL NUMBER OF PAGES UP TO GET ALL THE DATA
      const totalPages = Math.ceil(apiData.headers['x-wp-total']/99);

      for (let i = 1; i <= totalPages; i++) {
        promiseArray.push( request.get(`${request_url}&per_page=99&page=${i}`) )
      };

      resolve(
        Promise.all(promiseArray)
               .then((resolvedArray) => {

                 // PUSH IT ALL INTO A SINGLE ARRAY
                 let compiledPosts = [];

                 resolvedArray.forEach((axios_response) => {
                   // AXIOS MAKES US ACCESS W/RES.DATA
                   axios_response.data.forEach((post) => {
                     compiledPosts.push(post);
                   })
                 });

                 // RETURN AN ARRAY OF ALL POSTS REGARDLESS OF LENGTH
                 return compiledPosts;
               }).catch((e) => { console.log('ERROR'); reject(e);})
       )
    }
  ).catch((e) => { console.log('ERROR'); reject(e);})
})

Any creative ideas to make this pattern better?

Answer 1

I have exactly the same question. In my case, I use Vue Resource :

this.$resource('wp/v2/media').query().then((response) => {   

  let pagesNumber = Math.ceil(response.headers.get('X-WP-TotalPages'));

  for(let i=1; i <= pagesNumber; i++) {
       this.$resource('wp/v2/media?page='+ i).query().then((response) => {
           this.medias.push(response.data);
           this.medias = _.flatten(this.medias);
           console.log(this.medias);
       });
}

I'm pretty sure there is a better workaround to achieve this.