How to add header to downloadPage?

Question

How to add custom header ( for example header 'accept-encoding: gzip' ) for downloadPage in Twisted ?

Answer 1

If you look at the API documentation for downloadPage, you'll notice that it accepts *args and **kwargs. It refers you to HTTPDownloader for documentation regarding those parameters.

If you look at the API documentation for HTTPDownloader, you'll see that it accepts a headers argument. This argument isn't documented, but it's what you're looking for. Its value should be a dict mapping header name to header value. So,

downloadPage(..., headers={'accept-encoding': 'gzip'})

should do what you want. Also, note the newer client API provided by Twisted, twisted.web.client.Agent. Since Twisted 11.1, this supports content encodings at a higher level, and has specific support for gzip.