4

I've been trying to make a program that actively intercepts requests and returns the response body of those requests as a user browses a site (and performs requests).

It seems that the current code only intercepts the details of requests when it enters the site, but does not continue to print out details of requests as users perform actions on the site.

I would really appreciate any help!

Here is my entire program currently:

import asyncio
import json
import time
import os
from pyppeteer import launch
from datetime import datetime

args = [' --window-size=500,645']

async def intercept_network_response(response):
    # In this example, we care only about responses returning JSONs
    if "application/json" in response.headers.get("content-type", ""):
        # Print some info about the responses
        print("--------------------------------------")
        print("RESPONSE:", response)
        print("URL:", response.url)
        print("Method:", response.request.method)
        print("Response headers:", response.headers)
        print("Request Headers:", response.request.headers)
        print("Response status:", response.status)
        # Print the content of the response
        try:
            # await response.json() returns the response as Python object
            print("Content: ", await response.json())
            print("--------------------------------------")
        except json.decoder.JSONDecodeError:
            # NOTE: Use await response.text() if you want to get raw response text
            print("Failed to decode JSON from", await response.text())


async def main():
    print('Launching')
    browser = await launch(headless=False, autoclose=False, args=args)
    print('Opening')
    page = await browser.newPage()
    await page.setViewport({
        'width': 500,
        'height': 512
    })
    page.on('response', intercept_network_response)
    print('Going to site URL')
    await page.goto('https://kith.com/checkpoint')
    time.sleep(10000)
    await browser.close()


asyncio.get_event_loop().run_until_complete(main())
benastahl
  • 51
  • 4
  • When you say "as users perform actions on the site", what are you doing exactly? This code seems to be working fine. What did you do and what did you expect? If in `main()` you'd go to another page, like `await page.goto('https://kith.com/checkpoint2')` before the browser.close(), it would capture the response again? – JarroVGIT Feb 04 '21 at 23:30
  • I mean that when a person performs actions on the url `https://kith.com/checkpoint` like solve a captcha (which is what the page contains), it intercepts those requests in real time. Right now it seems it only intercepts requests made when going to the url `await page.goto('https://kith.com/checkpoint)`. – benastahl Feb 05 '21 at 06:34

0 Answers0