10

I am currently writing a HTTP client to do a HTTP POST on a URL that returns a HTTP response.

However, for error messages code 400 and 500, it sends back non chunked HTTP response, and for success messages, 201, it sends a chunked response.

In the request, I am setting the content-length, so I am not sure why it is still sending us the chunked transfer encoding. Is there any other header I can set in the request, that will tell the HTTP server not to send chunked encoding?

        headerList.append("POST /v2/charges HTTP/1.1")
        headerList.append("Content-Type: application/json")
        headerList.append("host: xxxxxxxxx")
        headerList.append("request-id: ABCD001123")
        headerList.append("Content-length: %d" %len(Msg))
        hostReqHeader = "\r\n".join(headerList)
        reqData = hostReqHeader + '\r\n\r\n' + qbPosMsg

I am using sockets to send these HTTP messages, and not using httplib or requests library.

roymustang86
  • 8,054
  • 22
  • 70
  • 101
  • Bad news: [RFC7230, ¶4.1](http://tools.ietf.org/html/rfc7230#section-4.1), "*A recipient MUST be able to parse and decode the chunked transfer coding.*" So, there is no general standard way prevent chunked encoding. I suppose there might be a way specific to your server and outside the scope of the standard. What server are you using? – Robᵩ Aug 12 '15 at 16:19
  • See also ¶4.3, "*A client MUST NOT send the chunked transfer coding name in TE; chunked is always acceptable for HTTP/1.1 recipients.*" – Robᵩ Aug 12 '15 at 16:23

2 Answers2

9

Chunked is a required feature of HTTP/1.1. If you do not require any other 1.1-specific features, specify HTTP/1.0 in your request:

    headerList.append("POST /v2/charges HTTP/1.0")
Robᵩ
  • 163,533
  • 20
  • 239
  • 308
6

The Content-Length header you are specifying in your request applies to the request, not the server's response.

Chunked transfer is only used by the HTTP/1.1 server in a response when the client specifies HTTP/1.1 as the protocol. If you want to disable chunked transfer completely, you specify HTTP/1.0 as the protocol in your request.

Alternatively, make use of an HTTP client library that supports chunked transfer - any one that supports HTTP/1.1 will, because in any HTTP/1.1 conversation the server is free to choose whether to use chunked transfer for any request.

HTTP/1.1 (and indeed HTTP/2) servers still support HTTP/1.0, and HTTP 1.0 remains quite useful for the ability to write simple clients with a few lines of code, that can still query modern web servers (albeit, with the need to TLS-wrap for HTTPS support). So it's quite appropriate in this situation. I think that is the beauty of HTTP, that the basic protocol is so simple. HTTP 1.1 and HTTP 2.0 add progressively more complexity in terms of being able to write a client that supports them, but all that complexity is optional - HTTP 1.0 can still be used.

thomasrutter
  • 114,488
  • 30
  • 148
  • 167
  • 1
    "Chunked transfer-encoding is never used for an HTTP request, only for a response, and only when the client specifies HTTP/1.1 as the protocol." - I do not think this is correct. The RFC refers to chunked encoding on a message body and makes no specific mention that it applies only to responses. https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1 – Niall Connaughton Nov 25 '21 at 01:25
  • 1
    Additionally, from the RFC - "All HTTP/1.1 applications MUST be able to receive and decode the "chunked" transfer-coding". Hence if your application does not support chunked encoding then by definition you're implementing HTTP 1.0, not 1.1. Which is fine - just tell the server you're 1.0. – Niall Connaughton Nov 25 '21 at 01:27
  • Or use version HTTP 2 – Obay Abd-Algader May 04 '22 at 09:00
  • 1
    I'd never considered chunked being used on the request but indeed looks like you're right. I guess you would need to be confident the server or proxy is HTTP/1.1 enabled, which you could determine out of band or determine based on previous server responses within the same connection being HTTP/1.1. These days HTTP/1.1 is the forgotten middle child in this regard as its keep alive and chunk transfers are abandoned in HTTP/2 for better multiplexing, and weren't supported in HTTP/1.0 – thomasrutter May 06 '22 at 01:26