2

I'm developing a screen scraping robot that uses Watir (ruby) to crawl specific web searches. Watir is used as the search results are delivered in pages, only available via AJAX requests.

My issue is now that to perform a new search, the browser has to be shut down in order for the search session to be cleared - otherwise the initial search overrule the change in the GET parameters.

Is it somehow possible to force Firefox to clear sessions on every request made?

Additionally, does anyone have experience solving these kind of issues via Watir?

TylerH
  • 20,799
  • 66
  • 75
  • 101
Jonas Bylov
  • 1,494
  • 2
  • 15
  • 28

2 Answers2

1

If the session is maintained via cookies in your firefox browser then it's possible.

All you have to remove the cookies from your firefox cookies repository before it starts. Firefox stores its cookies at (as of in my ubuntu and mac)

~/.mozilla/firefox/12wwonrk.default/cookies.sqlite  [in ubuntu]

or

~/Library/Application Support/Firefox/Profiles/eox4ghka.default/cookies.sqlite  [in mac]

(prior Firefox 3 it was cookies.txt instead sqlite)

If you can truncate the sqlite (or the txt) then the cookies will no longer be there.

As you are running Watir you are most probably using ruby. So, if you can run these commands through system or %x[] (or compatible commands through sqlite gem/lib) before Watir::Browser.new statement, hopefully you'll be done.

./sqlite3 path/to/cookies.sqlite
DELETE FROM moz_cookies;
.quit
intellidiot
  • 11,108
  • 4
  • 34
  • 41
0

If you want to use Watir, you can mess with profiles as described at http://watirwebdriver.com/. Most browsers seem to get their own profile for each new instance by default.

Sam King
  • 2,068
  • 18
  • 29