Stack Overflow
Stack Overflow

Questions tagged [httpx]

HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2.

HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2. For more details, please check the HTTPX website.

162 questions
2
votes
1 answer

Python httpx stream data asynchronously?

In the link https://zetcode.com/python/httpx/, it has the following example for stream import httpx url = 'https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-mini-memstick.img' with…
ca9163d9
  • 27,283
  • 64
  • 210
  • 413
2
votes
0 answers

Ways of sending data between FastAPI router methods / endpoints

Let's say we have two minimal Fast API endpoints: from urllib.parse import urlencode import httpx from fastapi import APIRouter router = APIRouter() @router.post("/api/endpoint_1") async def endpoint_1(attributes): # process the attributes …
Grajdeanu Alex
  • 388
  • 6
  • 20
2
votes
0 answers

Exception somewhere in the httpx/httpcore entrails

I've a scraping engine which uses any proxy list, and retries in case the proxy doesn't work. So there are plenty of proxies that timeout, have connection refused, bad certificates etc. After I switched to httpx from aiohttp I have plenty of…
aikipooh
  • 137
  • 1
  • 19
2
votes
1 answer

When using Python asyncio corurrent send network request, how to make coroutine prefer continue to handle response first, not send a new request?

If I want to request a API 1000 times (send network request and handle response), it will begin to handle response after sending all 1000 request first, and then handle response. Can I tell asyncio prefer to return back await position code if it's…
LudwigWS
  • 23
  • 5
2
votes
0 answers

Python requests/httpx repsonses and garbage collection

I originally found this using httpx, but it also applies to requests, which is more well known, so I'll be using the latter in my examples. I am making multiple requests in parallel, where the data is coming from a generator that creates byte chunks…
LoveToCode
  • 788
  • 6
  • 14
2
votes
1 answer

In aiohttp or httpx do I need to close session/client on application shutdown

I am using httpx library but I think the principle for aiohttp is the same. If I create and reuse AsyncClient for multiple requests throughout the lifetime of the application do I need to call aclose() (or close if use Client) at the application…
andnik
  • 2,405
  • 2
  • 22
  • 33
2
votes
1 answer

Parallel requests block infinitely after exactly 100 requests using asyncio

I've tried using both httpx and aiohttp, and both have this hard-coded limit. import asyncio import aiohttp import httpx async def main(): client = aiohttp.ClientSession() # client = httpx.AsyncClient(timeout=None) coros = [ …
Dev Aggarwal
  • 7,627
  • 3
  • 38
  • 50
1
vote
1 answer

Why the response of httpx is different compared to inspect element network?

I am trying to scrape this site https://shopee.co.id/S-B-Golden-Curry-Bumbu-Kari-Japanese-Curry-Mix-Medium-Hot-220-gr-i.65323877.3343602079?sp_atk=0b471505-cfca-491b-b7a2-453d455eecf6&xptdk=0b471505-cfca-491b-b7a2-453d455eecf6 I am using python…
Hal
  • 123
  • 1
  • 6
1
vote
2 answers

Selenium delay when python webscraping

I am running a web scraping task on Invest's website, the code below has always worked and started to give an error for a reason I am not identifying: from bs4 import BeautifulSoup from lxml import etree from urllib.request import Request,…
Leonardo Silva
  • 11
  • 1
1
vote
0 answers

Python HTTPX post Request timeout

We have a python 3.8 application that monitor others application and generate metrics for grafana Configuration: python 3.8, httpx 0.24.1 httpcore 0.17.2 Sometimes a check on applications failed with a request timeout exception: with httpx.Client()…
Vincent BILLARD
  • 11
  • 1
1
vote
1 answer

Iterating through requests until return is none (Python)

I am working with ticket inventory for various events that have an unknown total number of listings. For each listing, I get a return of some data such as price, quantity, owning user, etc. To collect this data I have created an async script that…
johne518
  • 77
  • 5
1
vote
1 answer

HTTPX RESPX Pytest TypeError: Invalid type for url. Expected str or httpx.URL, got :

I have a function in my Python class that works fine when I use it in my other .py file. @exception_handler def get_all_workspaces(self) -> Union[WorkspacesModel, GSResponse]: Client = self.http_client responses = Client.get(f"workspaces") …
krishna lodha
  • 381
  • 3
  • 9
1
vote
0 answers

HTTPX returns 403, but requests 200. Python

I'm trying to parse one site. I'm sending a naked http request, just by setting a User-Agent. It's surprising how a request using "requests" returns a 200 response. But the same query using httpx returns 403. I have tried making the request with…
Ivan
  • 37
  • 6
1
vote
0 answers

How to use csv.reader in an async context?

I query multiple services with asynchronously using httpx. These services return csv data that could be very large so I'm using streams. So far so good. The problem I'm having is that the Python standard csv.reader doesn't work with AsyncIterator's…
fanta fles
  • 149
  • 8
1
vote
0 answers

HTTPx Async Client not working when uploading file to FastAPI endpoint

I am using a HTTPx client. See this documentation for wrapping a FastAPI app in a HTTPx Async Client. https://fastapi.tiangolo.com/advanced/async-tests/#example It is just, not with block, I return the AsyncClient directly. Everything feels fine. I…
cansusargin
  • 31
  • 8
1 2
3
…
10 11