How do I add a header to urllib2 opener?

Question

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))    
opener.open('http://abc.com')
opener.open('http://google.com')

As you can see, I use opener to visit different websites, using a cookie jar. Can I set a header so that each time a website is it, the header is applied?

[What have you tried?](http://whathaveyoutried.com) – Piotr Dobrogost Sep 17 '12 at 08:39 — Piotr Dobrogost, Sep 17 '12 at 08:39

score 63 · Accepted Answer · answered Jun 07 '11 at 00:43

63

You can add the headers directly to the OpenerDirector object returned by build_opener. From the last example in the urllib2 docs:

OpenerDirector automatically adds a User-Agent header to every Request. To change this:

import urllib2
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
opener.open('http://www.example.com/')

Also, remember that a few standard headers (Content-Length, Content-Type and Host) are added when the Request is passed to urlopen() (or OpenerDirector.open()).

answered Jun 07 '11 at 00:43

senderle

145,869
36
209
233

6

@PiotrDobrogost, to be fair, this particular functionality is underdocumented. The [`OpenerDirector`](http://docs.python.org/release/2.6/library/urllib2.html#urllib2.OpenerDirector) class entry is bare; the above information is _very_ easy to miss. – senderle Sep 17 '12 at 12:07

Corey Goldberg · Answer 2 · 2012-09-17T02:31:15.417

18

headers = {'foo': 'bar',}
req = urllib2.Request(url, None, headers)
resp = urllib2.urlopen(req)

or

req = urllib2.Request(url)
req.add_header('foo', 'bar')
resp = urllib2.urlopen(req)

edited Sep 17 '12 at 02:31

answered Jun 07 '11 at 00:30

Corey Goldberg

59,062
28
129
143

2

`opener` is whole a different object, you can't use `Request` in this case. – ahmet alp balkan Aug 30 '12 at 07:17

How do I add a header to urllib2 opener?

2 Answers2

Linked

Related