10

Once that puppeteer goes to a certain url, I want that it listens to all the requests that are made, then find a specific request and return its response. The response should be a json object.

I managed in listening to all the requests and intercepting the desired one, but I don't know how to get its response. Here's my attempt: I get the error TypeError: Cannot read property 'then' of null.

Any suggestion? 

page.on('request',async(request)=>{
    console.log(request.url())

    if (request.url().includes('desiredrequest.json')){
        console.log('Request Intercepted')
        request.response().then(response => {
            return response.text();
        }).then(function(data) {
        console.log(data); // this will be a string
        alert(data)
        });
    }

    request.continue()
})
Drun
  • 529
  • 1
  • 7
  • 17
  • Does this answer your question? [How to get body / json response from XHR request with Puppeteer](https://stackoverflow.com/questions/56689420/how-to-get-body-json-response-from-xhr-request-with-puppeteer) – ggorlen Nov 24 '20 at 17:18

2 Answers2

17

Since the response may have not arrived yet, the better method would be listening on the response event and get the request object from it.

page.on('response', async(response) => {
    const request = response.request();
    if (request.url().includes('desiredrequest.json')){
        const text = await response.text();
        console.log(text);
    }
})
mbit
  • 2,763
  • 1
  • 10
  • 16
  • Since OP is looking for JSON, `response.json()` might be a more appropriate method. There's also [`page.waitForResponse`](https://stackoverflow.com/a/56839061/6243352) which lets you capture a specific response without a callback. – ggorlen Nov 24 '20 at 17:21
2

You might wanna use the "requestfinished" event instead of the "request" one.

page.on('requestfinished', async (request) => {
    const response = await request.response();

    const responseHeaders = response.headers();
    let responseBody;
    if (request.redirectChain().length === 0) {
        // Because body can only be accessed for non-redirect responses.
        if (request.url().includes('desiredrequest.json')){
            responseBody = await response.buffer();
        }
    }
    // You now have a buffer of your response, you can then convert it to string :
    console.log(responseBody.toString());

    request.continue()
});
Félix
  • 134
  • 8