Stack Overflow
Stack Overflow

Questions tagged [python-asyncio]

This tag is to be used for the asyncio Python package which provides mechanisms for writing single-threaded concurrent code. The asyncio package provides asynchronous I/O, event loop, coroutines and tasks beginning with Python 3.4.

This module provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives. Here is a more detailed list of the package contents:

  • a pluggable event loop with various system-specific implementations;
  • transport and protocol abstractions (similar to those in Twisted);
  • concrete support for TCP, UDP, SSL, subprocess pipes, delayed calls, and others (some may be system-dependent);
  • a Future class that mimicks the one in the concurrent.futures module, but adapted for use with the event loop;
  • coroutines and tasks based on yield from (PEP 380), to help write concurrent code in a sequential fashion;
  • cancellation support for Futures and coroutines;
  • synchronization primitives for use between coroutines in a single thread, mimicking those in the threading module;
  • an interface for passing work off to a threadpool, for times when you absolutely, positively have to use a library that makes blocking I/O calls.
  • Recently Python 3.5 introduced async/await for enhancing the module for more readablity instead of yield from.
7214 questions
3
votes
0 answers

Measure python asyncio task pressure

I have some python code that spins up quite a few asyncio tasks. I would like to know at which point I have spun up to many. Concretely, this means that I would like to measure how many tasks are currently waiting in a queue to be run (i.e., these…
Lasse
  • 221
  • 2
  • 3
3
votes
1 answer

Convert a simple multithreaded program with asyncio

I am still pretty new to Python asyncio, so I am trying to convert a simple problem I solved using multithreading, to using asyncio. I made an example of what I want to achieve. Each MiniBot instance can start at random times (those time.sleep()…
Uno
  • 61
  • 3
3
votes
1 answer

Handling a lot of concurrent connections in Python 3 asyncio

Iam trying to improve the performance of my application. It is a Python3.6 asyncio.Protocol based TCP server (SSL wrapped) handling a lot of requests. It works fine and the performance is acceptable when only one connection is active, but as soon…
pistol-whip
  • 41
  • 1
  • 5
3
votes
1 answer

Python: How to use asyncio with huge csv-files to send asynchronous requests from loops?

I want to go through a huge list of urls and send requests to them asynchronously. Since the CSV-file with the urls is too big to load it at once, I would like to read the lines row by row, and each time the row is loaded, it should start a request…
Felix
  • 85
  • 2
  • 6
3
votes
1 answer

General solution to offloading blocking function to executor in Tornado app

I am trying to find a general solution for offloading blocking tasks to a ThreadPoolExecutor. In the example below, I can achieve the desired non-blocking result in the NonBlockingHandler using Tornado's run_on_executor decorator. In the asyncify…
jdesilvio
  • 1,794
  • 4
  • 22
  • 38
3
votes
1 answer

Do AsyncIO stream writers/readers require manually ensuring that all data is sent/received?

When dealing with sockets, you need to make sure that all data is sent/received, since you may receive incomplete chunks of data when reading. From the docs: In general, they return when the associated network buffers have been filled (send) or…
Carcigenicate
  • 43,494
  • 9
  • 68
  • 117
3
votes
1 answer

Asyncio function works when called from script but not from Flask route

I'm a novice with Python and these libraries/modules. I'm writing a simple ping-test network scanner as a learning project. I first developed a script using asyncio to ping addresses on a network #ip_test.py import asyncio import ipaddress async…
Charlie Patton
  • 435
  • 2
  • 12
3
votes
1 answer

Why is reading and calling an API from a file slower using Python async than synchronously?

I have a large file, with a JSON record on each line. I'm writing a script to upload a subset of these records to CouchDB via the API, and experimenting with different approaches to see what works the fastest. Here's what I've found to work fastest…
James Kelleher
  • 1,957
  • 3
  • 18
  • 34
3
votes
1 answer

How to read from socket using asyncio add_reader

I have this code: import sys import socket import asyncio async def main(dest_addr, max_hops=30, timeout=0.5): loop = asyncio.get_event_loop() queue = asyncio.Queue() port = 33434 rx = socket.socket(socket.AF_INET,…
NicoAdrian
  • 946
  • 1
  • 7
  • 18
3
votes
4 answers

How to aiohttp request post files list python requests module?

I want to multi post using aiohttp. And, I need post with FILE. But, my code dosen't work This is my code import aiohttp file = open('file/path', 'rb') async with aiohttp.request('post', url, files=file) as response: return await…
lee
  • 53
  • 1
  • 2
  • 5
3
votes
2 answers

How can i use python coroutines as celery tasks

I have old celery version in my tornado project. My current celery version is 3.1.23. There was an old library https://github.com/mher/tornado-celery which is dead now, so i would like to get rid of it and update celery to new version. I don't want…
Alexandr Zayets
  • 299
  • 1
  • 8
3
votes
1 answer

How to use multiprocessing.Event in an asyncio event loop?

I am using the multiprocessing module to create a Process. In this child process, it will run an asyncio event loop, and in the parent process, I am not. How can I use syncronization primitives for this pair of processes? Let's say I want a way for…
dzil123
  • 47
  • 1
  • 4
3
votes
1 answer

Nats Subscriber continuously listening to publisher

I'm trying to get a NATS subscriber to continuously listen to published messages. My Publisher is an API endpoint which can be hit in a browser. My Subscriber is a python app which should run forever, listening for published messages. My problem is…
user4292309
3
votes
1 answer

Asynchronically wait for a method to complete in Python 3

Consider this simplified example: def get_external_thing() -> str: my_variable = a_blocking_operation() my_variable.some_other_operation() return my_variable externally_gathered_thing = None while True: sleep(1) …
Artur
  • 973
  • 1
  • 14
  • 29
3
votes
2 answers

Future from asyncio.run_coroutine_threadsafe hangs forever?

As a followup to my previous question about calling an async function from a synchronous one, I've discovered asyncio.run_coroutine_threadsafe. On paper, this looks ideal. Based on the comments in this StackOverflow question, this looks ideal. I can…
cf-
  • 8,598
  • 9
  • 36
  • 58
1 2 3
…
99
100