0

I am writing a script that automates the completion a web form in my Rails app using the form entries given on the client side. However, this site uses Javascript, and so Mechanize is out of the question.

However, everything I've read about Mechanize's alternatives -- Watir Webdriver, Selenium, Capybara Webkit -- all focus seemingly exclusively on testing. However, my Rails web app would take in form entries from users, and then enter them using one of these tools into another website. For example, I would need to upload an image (ie :image) and enter in different text (ie :city) into form fields as part of this app, which would take the entries and enter them into the website.

So my first question: Can I use any Mechanize alternatives for something besides testing? And second: Can anyone refer to code examples on the web for non-testing usages of any of the above automators?

CodeBiker
  • 2,985
  • 2
  • 33
  • 36

3 Answers3

1

I don't have any concrete examples of javascript-enabled alternatives used in non-testing contexts, but I do have a suggestion: if you know the website that you will be submitting the form info to, it's probably better to find out what the javascript is doing and mimic that instead. Dig into the site's javascript code and figure out what type of data is being submitted to what URL, and just mimic that using standard HTTP operations -- skip the javascript rendering/interaction part altogether.

There is a lot of overhead incurred when rendering a page with javascript, which is why these tools (Watir, Selenium, Capybara and the like) are not generally used in actual client-facing application contexts.

Chris Salzberg
  • 27,099
  • 4
  • 75
  • 82
  • I didn't realize that it's possible to work around the Javascript with Mechanize. I'll try that out. – CodeBiker Jul 19 '13 at 18:44
  • 1
    It's possible to work around the javascript with anything. Remember that javascript in the end is just a wrapper executing HTTP requests just like a normal HTML form would, so if you can figure out what those requests are, you can reproduce them and bypass that client-side layer altogether. A form, for example, will typically submit a `POST` request with some payload -- if you know the payload and the URL to send it to, you can send it directly. Use firebug or a similar debugging/inspector tool to track down what's going on behind the scenes. – Chris Salzberg Jul 20 '13 at 02:01
  • I've been working on this for a while but have run into trouble: I have an image that I need to click on, but its link is `javascript:void(false);` and an id tag `img1`. I can't figure out how to circumvent this. I tried looking at the Javascript code, which doesn't seem to indicate how I can get around it either. And according to [this](http://stackoverflow.com/questions/11895973/clicking-image-with-mechanize), it shouldn't be possible. If you know of any resources that could help me figure this out, that'd be great. – CodeBiker Jul 22 '13 at 06:57
  • Why do you have to click on the image? If the image is a submit button on a form, then just submit the form. If it saves the image filename somewhere to send later, then figure out how to get the filename and send it just like the javascript would. Ultimately you don't care about the clicking, you just care about what HTTP actions are triggered as a result. `javascript:void(false):` btw is just telling the browser not to perform whatever default action it would normally perform, ref: http://stackoverflow.com/questions/3498492/javascriptvoid0-vs-return-false-vs-preventdefault – Chris Salzberg Jul 22 '13 at 08:09
  • Clicking on the image makes a slider move over to the left, which as I understand it is not an HTTP request. I've scanned over much of the very lengthy Javascript for a couple hours, and I unfortunately can't find an indicator of how to work around it. – CodeBiker Jul 22 '13 at 15:07
  • But my point is: what effect does that slider ultimately have on the HTTP request you send? What are you adjusting with the slider? Whatever it is, it must affect some information sent to the server, right? So what is that information? Don't worry about actually trying to adjust the slider, there's no point. – Chris Salzberg Jul 22 '13 at 22:30
0

Watir has a headless gem. You can give it a try watir headless

az7ar
  • 5,187
  • 2
  • 19
  • 23
  • I do know about watir's capability with headless, but unless I'm mistaken, it doesn't seem capable for non-testing purposes either. – CodeBiker Jul 19 '13 at 14:11
0

You should be able to use watir-webdriver to take the data (image, city) from one site and upload to other site. Below is brief code sample to help you get started. 

require 'watir-webdriver'
$browser1 = Watir::Browser.new : chrome #You can use phantomjs for headless http://phantomjs.org/ 
$browser1.goto  http://website1.com
city_field = $browser1.text_field (:id => 'city')
city = city_field.value 
$browser2 = Watir::Browser.new : chrome 
$browser2.goto  http://website2.com
city_field_site2 = $browser2.text_field (:id => 'city')
city_field_site2.set city
Arpit
  • 1
  • Interesting -- I didn't realize watir-webdriver had that capability. I am looking to use my own Rails web form as a way for users to input their data (ie city and image), and then the script would enter and upload that information onto an external website. From reading this, it seems like that should be possible; that is, that the script can use data inputted by the user instead of test data populated by the programmer. – CodeBiker Jul 21 '13 at 03:23
  • It should work. You just have to ensure you are using the right accessors for both browser instances. Let me know if you face any issues. – Arpit Jul 22 '13 at 03:07