Is http stream actually implemented by http chunk?

Question

At first, I thought http stream is actually implemented http chunk.

So I did a test to learn.

Here is a django view

def test_stream(request):

    return StreamingHttpResponse(func)

func return iterable Here is the output using curl to access the view

curl -vv -raw http://172.25.44.238:8004/api/v1/admin/restart_all_agent?cluster_id='dd5aef9cbe7311e99585f000ac192cee' -i
Warning: Invalid character is found in given range. A specified range MUST 
Warning: have only digits in 'start'-'stop'. The server's response to this 
Warning: request is uncertain.
* About to connect() to 172.25.44.238 port 8004 (#0)
*   Trying 172.25.44.238...
* Connected to 172.25.44.238 (172.25.44.238) port 8004 (#0)
> GET /api/v1/admin/restart_all_agent?cluster_id=dd5aef9cbe7311e99585f000ac192cee HTTP/1.1
> Range: bytes=aw
> User-Agent: curl/7.29.0
> Host: 172.25.44.238:8004
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
* no chunk, no close, no size. Assume close to signal end

< 

some http response body.

* Closing connection 0

From the output you can see there is no chunked header. It seems Http stream has nothing to do with chunk.

So here is the question

Is http stream implemented by http chunk?
How to return a chunked response in django?

incorrect quoting, move the first apostrophe to the start of http:// — hanshenrik, Dec 16 '19 at 08:20
The implementation is actually done by your wsgi server, what are you using to test this (if mod_wsgi, which version)? And func() should `yield`. — dirkgroten, Dec 16 '19 at 15:45
@dirkgroten yes. func return iterable and I am using uwsgi 2.0.18 to run django. — Kramer Li, Dec 19 '19 at 02:16
The argument `-raw` should probably be `--raw` (two dashes). `-raw` is `-r` (range) with an invalid range (`aw`) as `curl` also warns before sending the request. — axiac, Dec 25 '19 at 22:17

score 2 · Answer 1 · answered Dec 19 '19 at 11:20

No, streaming does not have anything to do with chunking. You can have one with or without the other.

The advantage of a streaming response is that you aren't required to load the whole response into memory as a string at once.

This blog article does a decent job at explaining what the use case is: https://andrewbrookins.com/django/how-does-djangos-streaminghttpresponse-work-exactly/

It also covers what's needed to return a chunked response, which I'll summarize:

Use StreamingHttpResponse class, but provide an iterable (list, generator...) as the first argument rather than a string.
Do not set a Content-Length header.
You need to use a webserver which chunks the response for you. Django itself does not do this, and it will not happen with manage.py runserver (but uvicorn and gunicorn do, for example)

from django.urls import path
from django.http import StreamingHttpResponse

def generator():
    for i in range(0, 100):
        yield str(i) * 1024 * 1024

def chunked_view(request):
    return StreamingHttpResponse(generator(), content_type='text/plain')

urlpatterns = [path('chunked', chunked_view)]

Run the Django application using uvicorn myapp.asgi:application.

curl -v http://127.0.0.1:8000/chunked > /dev/null shows transfer-encoding: chunked in the response headers.

"The advantage of a streaming response is that you aren't required to load the whole response into memory as a string at once." Chunking also have this advantage since the response are returned to client by pieces. So why we need chunking if we areadly have stream? Or what is the difference of these two technology? — Kramer Li, Dec 26 '19 at 02:27

Is http stream actually implemented by http chunk?

1 Answers1