Stack Overflow
Stack Overflow

Questions tagged [mechanize]

Mechanize is a library for automated web browsing originally developed for Perl, there are now also Python and Ruby implementations.

Mechanize is a Ruby library for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history. It is adapted from the Perl module. There is also a for Python.

2512 questions
25
votes
7 answers

mechanize select form using id

I am working on mechanize with python.
The form here does not have a name. How can I parse the form using it's id?
sam
  • 18,509
  • 24
  • 83
  • 116
24
votes
3 answers

How to get the page source with Mechanize/Nokogiri

I'm logged into a webpage/servlet using Mechanize. I have a page object: jobShortListPg = agent.get(addressOfPage) When I use: puts jobShortListPg I get the "mechanized" version of the page which I don't want: #
Waley Chen
  • 929
  • 3
  • 10
  • 23
24
votes
6 answers

Are there any alternatives to Mechanize in Python?

I'm using Python 3.6 while I have to fill in a form. Unfortunately, mechanize doesn't work on Python 3. What do you suggest as an alternative to mechanize?
user319723
  • 451
  • 1
  • 5
  • 8
23
votes
2 answers

Error - urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

My aim is to extract the html from all the links in the first page after entering the google search term. I work behind a proxy so this is my approach. 1.I first used mechanize to enter the search term in the form , ive set the proxies and robots…
Manoj
  • 961
  • 4
  • 11
  • 37
22
votes
4 answers

Python mechanize, following link by url and what is the nr parameter?

I'm sorry to have to ask something like this but python's mechanize documentation seems to really be lacking and I can't figure this out.. they only give one example that I can find for following a link: response1 =…
Rick
  • 16,612
  • 34
  • 110
  • 163
22
votes
2 answers

Installing mechanize for python 3.4

I'm trying to retrieve the mechanize module for python 3.4. Can anybody guide me in the right direction and perhaps walk me through the steps that I would need to take in order to make the correct installation? I'm currently using Windows 10.
Carnageta
  • 321
  • 1
  • 3
  • 15
22
votes
3 answers

Ruby Mechanize: user agents?

How many user agents are there in Mechanize? Is there a handy list of all the user agent options anywhere?
trnsfrmr
  • 223
  • 1
  • 2
  • 5
22
votes
5 answers

How to manually add a cookie to Mechanize state?

I'm working in Ruby, but my question is valid for other languages as well. I have a Mechanize-driven application. The server I'm talking to sets a cookie using JavaScript (rather than standard set-cookie), so Mechanize doesn't catch the cookie. I…
fearless_fool
  • 33,645
  • 23
  • 135
  • 217
22
votes
2 answers

How to login to a website with python and mechanize

i'm trying to log in to the website http://www.magickartenmarkt.de and do some analyzing in the member-area (https://www.magickartenmarkt.de/?mainPage=showWants). I saw other examples for this, but i don't get why my approaches didn't work. I…
Rappel
  • 696
  • 2
  • 6
  • 14
19
votes
1 answer

Selenium Webdriver vs Mechanize

I am interested in automating repetitive data entry in some forms for a website I frequent. So far the tools I've looked up that would provide support for this in a headless fashion could be Selenium WebDriver and Mechanize. My question is, is…
randombits
  • 47,058
  • 76
  • 251
  • 433
19
votes
3 answers

I get an error in python3 when importing mechanize

I get an error in python3 when importing mechanize. I've just installed mechanize into my virtualenv where python3 is installed. $ which python3 /Users/myname/.virtualenvs/python3/bin/python3 $ pip freeze mechanize==0.2.5 But, when I try to import…
crzyonez777
  • 1,789
  • 4
  • 18
  • 27
19
votes
4 answers

How to test a ruby application which uses mechanize

I wrote a small program that uses Mechanize to traverse a site. I want to write tests for it, but don't want it to actually go log onto the site every time I run the tests. I would like to mock the internet, so that when it goes to some site, it…
Joshua Cheek
  • 30,436
  • 16
  • 74
  • 83
19
votes
2 answers

Mechanize and BeautifulSoup for PHP?

I was wondering if there was anything similar like Mechanize or BeautifulSoup for PHP?
Alix Axel
  • 151,645
  • 95
  • 393
  • 500
18
votes
2 answers

Using WWW:Mechanize to download a file to disk without loading it all in memory first

I'm using Mechanize to facilitate the downloading of some files. At the moment my script uses the following line to actually download the files... agent.get('http://example.com/foo').save_as 'a_file_name' However this downloads the complete file…
Thomas
  • 181
  • 1
  • 1
  • 3
18
votes
3 answers

CertificateError: hostname doesn't match

I'm using a proxy (behind corporate firewall), to login to an https domain. The SSL handshake doesn't seem to be going well: CertificateError: hostname 'ats.finra.org:443' doesn't match 'ats.finra.org' I'm using Python 2.7.9 - Mechanize and I've…
pugmastaflex
  • 470
  • 1
  • 5
  • 17
1
2
3
…
99 100