35

I’m going through the documentation for the Selenium WebDriver, and it can drive Chrome for example. I was thinking, wouldn't it be far more efficient to ‘drive’ PhantomJS?

Is there a way to use Selenium with PhantomJS?

My intended use would be web scraping: The sites I scrape are loaded with AJAX and lots of lovely JavaScript, and I’m thinking this setup could be a good replacement for the Scrapy Python framework that I’m currently working with.

dakab
  • 5,379
  • 9
  • 43
  • 67
Joseph
  • 3,899
  • 10
  • 33
  • 52
  • I notice there is an internal class `PhantomJSDriver : RemoteWebDriver, ITakesScreenshot` but you cant create an instance of it (yet) – Mark Broadhurst Dec 07 '12 at 11:49

2 Answers2

56

PhantomJS now includes the GhostDriver project.

You are also suggested to use PhantomJS directly or with a convenience library such as CasperJS. CasperJS is specifically designed to make it easy to do sequential operations to web pages, perfect for many automation tasks.

Disclaimer: I am the author of PhantomJS.

Edit: As noted in Nick's answer, GhostDriver is now included in PhantomJS.

Steve HHH
  • 12,947
  • 6
  • 68
  • 71
Ariya Hidayat
  • 12,523
  • 3
  • 46
  • 39
  • Oh wow Ariya responded, I was reading some of your stuff the other day, I really admire your work. This is kinda awesome, anyways, if you don't mind, whats the best way to incorporate PhantomJS/CasperJS into a application. I was thinking of using Selenium because I want the whole program to loop and run continually (its real time and slightly interactive) When I was going throgh Phantom documentation I got the impression it's a command line tool for one off uses, and calling it from python would allow me to make it into a full program. Any tips? – Joseph Jun 27 '12 at 15:41
  • 5
    You can take some inspirations from a lot of PhantomJS user articles (check the wiki). – Ariya Hidayat Jul 01 '12 at 16:50
  • The wiki is empty. Any clues where I can get help on making this work with PHP? – Hippyjim May 12 '13 at 16:38
  • @Joseph: regarding the 'one off uses' issue, I also ran into that and created PhearJS for that (https://github.com/Tomtomgo/phearjs). It wraps a pool of PhantomJS instances and provides you with an API to it. – TTT Aug 18 '15 at 09:29
20

@Joseph, since the 1.8 release GhostDriver is included in the stable release of PhantomJS. Here is the exact release notes: http://phantomjs.org/release-1.8.html. You can simply start PhantomJS process to listen on some port, like this:

phantomjs --webdriver=PORT

Kudos to @detro and PhantomJS team for awesome work!

Nik
  • 1,508
  • 1
  • 13
  • 15