-1

I would like to use this code with a headless browser (e.g. phantomjs or casperjs) and then download the csv file at the end (instead of the native browser's saver popup dialog).

library("RSelenium")
startServer()
mybrowser <- remoteDriver(browserName = 'chrome')
mybrowser$open()
mybrowser$navigate("http://steamspy.com/login/")
wxbox <- mybrowser$findElement(using = 'xpath', "//*/input[@name = 'username']")
wxbox$sendKeysToElement(list("myusername"))
wxbox <- mybrowser$findElement(using = 'xpath', "//*/input[@name = 'password']")
wxbox$sendKeysToElement(list("mypassword"))
wxbutton <- mybrowser$findElement(using = 'xpath', "//*/button[@name = 'submit']")
wxbutton$clickElement()
mybrowser$navigate("http://steamspy.com/sale.php?tagid=0&from=2015-04-01&to=2016-04-25&submit=")
wxbutton <- mybrowser$findElement(using = 'css selector', "#ToolTables_tablesales_0")
wxbutton$clickElement()

Simply by changing chrome with phantomjs leads to this following error at opening and navigating the page:

Error:   Summary: UnknownError
     Detail: An unknown server-side error occurred while processing the command.
     class: org.openqa.selenium.UnsupportedCommandException
MrFlick
  • 195,160
  • 17
  • 277
  • 295
noblabla
  • 113
  • 1
  • 13

1 Answers1

0

Install PhantomJS if you haven't done this: Go Here. Download the windows zip file and extract it;

Add the path to phantoms.exe which is usually in the extractedFolder\bin to the environment variable path by going to Control Panel -> System -> advanced system settings -> environment variables, and add the path to the PATH variable, ignore this if you know how to this already;

Restart your RStudio or R session;

Run the following:

library(RSelenium)
pJS <- phantom()
myBrowser <- remoteDriver(browserName = "phantomjs")
myBrowser$open()
Psidom
  • 209,562
  • 33
  • 339
  • 356
  • PhantomJS doesn't see element that Chrome does. I get `Error: Summary: NoSuchElement Detail: An element could not be located on the page using the given search parameters. class: org.openqa.selenium.NoSuchElementException` at `wxbutton <- mybrowser$findElement(using = 'css selector', "#ToolTables_tablesales_0")` – noblabla Apr 27 '16 at 06:54
  • By `wxbutton <- mybrowser$findElement(using = 'css selector', "#ToolTables_tablesales_0")`, which element are you trying to find? – Psidom Apr 27 '16 at 13:02
  • The csv download button, it only appears once one is logged in. PhanthomJS may not save the authentication state before moving into the new page. – noblabla Apr 27 '16 at 13:07
  • It should normally since it is a browser. Did you replaced the username and password with you login information? – Psidom Apr 27 '16 at 13:13
  • And also try `mybrowser$setWindowSize(1400, 1000)` before trying to find the element. – Psidom Apr 27 '16 at 13:14
  • Yes, I replaced them. I find the behavior with phantomjs unconsistent, at some runs I get ` Summary: InvalidElementState Detail: An element command could not be completed because the element is in an invalid state (e.g. attempting to click a disabled element). class: org.openqa.selenium.InvalidElementStateException ` at `wxbox$sendKeysToElement` – noblabla Apr 27 '16 at 13:31
  • `mybrowser$setWindowSize(1400, 1000)` doesn't solve the problem – noblabla Apr 27 '16 at 13:37
  • Set a `Sys.sleep(10)` between `navigate` and `findElement`. You need to set some time for the browser to load the page. Set `10` accordingly it is the estimated seconds for loading the page. – Psidom Apr 27 '16 at 13:37
  • It doesn't work with `sys.sleep`. I notice RStudio command line doesn't immediately give the prompt back `>` while executing `navigate`. I assume it's loading the long page at that moment. – noblabla Apr 27 '16 at 13:49