Python Asyncio in Django View

Question

I would like to make two POST requests from an API on a Django view at the same time.

This is how I would do it outside of django.

import asyncio
import speech_recognition as sr

async def main(language1, language2):
    loop = asyncio.get_event_loop()
    r = sr.Recognizer()
    with sr.AudioFile(path.join(os.getcwd(), "audio.wav")) as source:
        audio = r.record(source)
    def reco_ibm(lang):
        return(r.recognize_ibm(audio, key, secret language=lang, show_all=True))
    future1 = loop.run_in_executor(None, reco_ibm, str(language1))
    future2 = loop.run_in_executor(None, reco_ibm, str(language2))
    response1 = await future1
    response2 = await future2

loop = asyncio.get_even_loop()
loop.run_until_complete(main("en-US", "es-ES"))

I'm confused about the event loop. How can I do this inside my Django view? Do I need to use nested functions for this?

def ibmaudio_ibm(request, language1, language2):
     #Asyncio code here

Edit: How is this even considered a duplicate? Parallel calls and schedulling with crontab are completely different things...

Possible duplicate of [using asyncio to do periodic task in django](https://stackoverflow.com/questions/43838872/using-asyncio-to-do-periodic-task-in-django) — e4c5, Jun 21 '17 at 14:54
@e4c5 It has nothing to do with schedulling tasks, I want parallel calls... Not even close to a duplicate — Juanvulcano, Jun 21 '17 at 17:58
Don't be mislead by the title. The question and the answer are the same — e4c5, Jun 21 '17 at 22:26

Juanvulcano · Accepted Answer · 2017-06-21T18:21:31.290

25

Solution was to nest the function inside another one.

def djangoview(request, language1, language2):
    async def main(language1, language2):
        loop = asyncio.get_event_loop()
        r = sr.Recognizer()
        with sr.AudioFile(path.join(os.getcwd(), "audio.wav")) as source:
            audio = r.record(source)
        def reco_ibm(lang):
            return(r.recognize_ibm(audio, key, secret language=lang, show_all=True))
        future1 = loop.run_in_executor(None, reco_ibm, str(language1))
        future2 = loop.run_in_executor(None, reco_ibm, str(language2))
        response1 = await future1
        response2 = await future2
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main(language1, language2))
    loop.close()
    return(HttpResponse)

edited Jun 21 '17 at 18:21

answered Jun 21 '17 at 18:01

Juanvulcano

1,354
3
26
44

3

this is still blocking for the django request response cycle – Ayush Jun 03 '18 at 20:29
@AyushShanker Can you further explain this? i was able to reduce the running time of my project quite a lot using this approach, I was waiting to receive output from API – Juanvulcano Jun 04 '18 at 06:36
7

@Juanvulcano it works faster because you run async requests concurrently. However, `loop.run_until_complete(main(language1, language2))` blocks further execution until async actions are complete. That means django worker won't be able to serve other requests till this one is complete. – Victor Miroshnikov Jun 06 '18 at 10:43
What should i do to not block django while using the above approach? For example after receiving the request return the httpResponse and let the task run in background? – Yash Agrawal Feb 15 '19 at 10:18
1

It's working for me and boost up my server speed to 5s from 78s for 1000 requests. – Harun-Ur-Rashid May 08 '19 at 06:58

Bob · Answer 2 · 2021-03-10T18:36:16.067

In this particular case you can simply use the ThreadPoolExecutor, asyncio is using it under the hood in .run_in_executor anyway (but also adds redundant lines of code / loop creation etc in your example).

# or likely ProcessPoolExecutor is better for cpu-heavy work
from concurrent.futures import ThreadPoolExecutor, wait

# create the executor outside of the view with the number of workers you may need
executor = ThreadPoolExecutor(max_workers=2)

def reco_ibm(lang):
    return(r.recognize_ibm(audio, key, secret language=str(lang), show_all=True))

def djangoview(request, language1, language2):
    r = sr.Recognizer()
    with sr.AudioFile(path.join(os.getcwd(), "audio.wav")) as source:
        audio = r.record(source)
        
        # then use it pretty trivially:
        futures = []
        for lang in [language1, language2]:
            futures.append(executor.submit(reco_ibm, lang)
        completed, pending = wait(futures)
        # `pending` will always be empty here (see the docs on wait)

        result1, result2 = [i.result() for i in completed]

    # do whatever you want with results etc.

see https://docs.python.org/3/library/concurrent.futures.html

This worked easily, small spelling mistake on line `result1, result2 = [i.resut() for i in completed]` — Alfa Bravo, Mar 10 '21 at 10:06

score 2 · Answer 3 · answered Jun 21 '17 at 05:23

2

Django is a synchronous framework so you can't use any async/await in the views because of there no loop or something like that.

You really can use Django channels library for it, but it will make your views asynchronous under the hood by itself, you don't need to use async also, just connect the channels a go on coding as you do it before, without any async features.

answered Jun 21 '17 at 05:23

vZ10

2,468
2
23
33

4

This is wrong. If you don't use async/await (or the old yield from etc syntax) in the code it will always be synchronous as there will be no switch points (i.e. await expressions). Even with Channels, as the Channels library doesn't change the way python work (unless you use something like gevent / twisted etc). – Bob Jul 03 '18 at 11:06

score 1 · Answer 4 · edited Dec 13 '21 at 15:35

You can try my way:

At first you need to create app.py in your app_name folder

Then fill your code:

async def main(language1, language2):
     r = sr.Recognizer()
     with sr.AudioFile(path.join(os.getcwd(), "audio.wav")) as source:
         audio = r.record(source)
def reco_ibm(lang):
    return r.recognize_ibm(audio, key, secret)
future1 = loop.run_in_executor(None, reco_ibm, str(language1))
future2 = loop.run_in_executor(None, reco_ibm, str(language2))
response1 = await future1
response2 = await future2

in app_name folder fill your views.py:

import asyncio
from django.http import HttpResponse
from . import app


# Create your views here.
def index(request):
    loop = asyncio.new_event_loop()
    ss = loop.run_until_complete(app.main(language1, language2))
    loop.close()
    return HttpResponse(ss, content_type='text\plain')

Python Asyncio in Django View

4 Answers4

Linked