Mechanize submit a form that creates a dynamic flie that needs to be downloaded

Question

So far I have mechanize code that does this:

goes to a site
logs in
submits a form

heres where i hit problems. What I need it to do is to write the response (a file) to a local file. I am pretty clueless as far as python interacting with the file system.

Thanks in advance

EDIT: Here is some of the code i currently have

br = mechanize.Browser()
br.set_handle_robots(False)
br.set_handle_redirect(True)
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1000)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]

formcount=0
for frm in br.forms():  
  if str(frm.attrs["id"])=="id-of-form":
    break
  formcount=formcount+1
br.select_form(nr=formcount)

with open('a filename', 'wb') as f:
    shutil.copyfileobj(br.submit(name='submit', label='value of submit button'), f)

If it matters; I'm running mac OS X

icktoofay · Accepted Answer · 2013-05-19T01:57:28.267

2

The return value of submit is a file-like object. You can copy the data to a local file:

import shutil
with open('downloaded', 'wb') as f:
    shutil.copyfileobj(br.submit(), f)

Unrelatedly, you can shorten the form selection bit like this:

br.select_form(predicate=lambda form: form.attrs['id'] == 'id-of-form')

Here's a full working example:

import mechanize
import shutil

br = mechanize.Browser()
br.open('http://stackoverflow.com/')
br.select_form(predicate=lambda form: form.attrs.get('id') == 'search')
br['q'] = '[python-mechanize]'
with open('search results.html', 'wb') as f:
    shutil.copyfileobj(br.submit(), f)

edited May 19 '13 at 01:57

answered May 19 '13 at 01:41

icktoofay

126,289
21
250
231

I think I understand what that is; but could you show where you would put the form select and the submitting of the form...or is the `br.submit()` submitting the form – zggz12 May 19 '13 at 01:44
@zggz12: `br.submit()` submits the form. – icktoofay May 19 '13 at 01:45
then the selecting of the form is done prior to this and also where is it that i tell it what file to write to – zggz12 May 19 '13 at 01:46
@zggz12: Yes, you fill out the form before this. The filename to write to is the first argument to `open`, so you'd replace `'downloaded'` with whatever expression you want. – icktoofay May 19 '13 at 01:49
i added that code to the end of the script...and i can't tell what's going on; i'm gonna edit my question to make it more specific – zggz12 May 19 '13 at 01:50
now that I look at my code, I'm not sure if I'm submitting the form....is it possible to print the whole html content of the selected form? – zggz12 May 19 '13 at 01:59
@zggz12: I can't see a way to do that, no, but if it helps you can check the form's `action` or `method`, e.g., `print br.form.action`. – icktoofay May 19 '13 at 02:07
I got it working... I'm not sure what I did differently; but it works...Thanks @icktoofay – zggz12 May 19 '13 at 03:17

Mechanize submit a form that creates a dynamic flie that needs to be downloaded

1 Answers1