RSelenium: clicking a link within another link

Question

I have this RSelenium script:

library(tidyverse)
library(RSelenium) # running through docker
library(rvest)
library(httr)

remDr <- remoteDriver(port = 4445L, browserName = "chrome")
remDr$open()


remDr$navigate("https://books.google.com/")
books <- remDr$findElement(using = "css", "[name = 'q']")

books$sendKeysToElement(list("NHL teams", key = "enter"))

bookElem <- remDr$findElements(using = "xpath",
                               "//h3[@class = 'LC20lb']//parent::a")

links <- sapply(bookElem, function(bookElem){
  bookElem$getElementAttribute("href")
})

The above clicks every link on the Google search return (there are 10 per page). The books that I search mostly have a preview once you click into it. If there's a preview, there's a small About this book link to click on that brings you to the publishing info.

I want to click on the first links, and then if there's a preview, click on "About this book." I have the below, but I just get Error: object of type 'closure' is not subsettable errors:

for(link in links) {

  # Navigate to each link
  remDr$navigate(link)

  # If statement to get past book previews
  if (str_detect(link, "frontcover")) {

   link2 <- remDr$findElement(using = 'xpath', 
                               '//*[@id="sidebar-atblink"]//parent::a')
   link2 <- as.list(link2)
   print(class(link2))
   link2_about <- sapply(link2, function(ugh){
      ugh$getElementAttribute('href')
    })

  } else {
    print("nice going, dumbass")
  }
}

Or I try a for loop instead of a the sapply, I get Error: $ operator is invalid for atomic vectors:

for(link in links) {

  # Navigate to each link
  remDr$navigate(link)

  # If statement to get past book previews
  if (str_detect(link, "frontcover")) {

    link2 <- remDr$findElement(using = 'xpath',
       '//a[@id="sidebar-atb-link" and span[.="About this book"]]')

     for(i in length(link2)){
      i$getElementAttribute('href')
     }

    } else {
     print("dumbass")
   }
}

How can I successfully click into that second link, depending on if the preview is there? Thanks!

supputuri · Accepted Answer · 2019-03-22T19:08:28.190

1

Just update the below line.

aboutLinks <- remDr$findElements(using = 'xpath', 
                           '//a[@id="sidebar-atb-link" and span[.="About this book"]]')
links2 <- sapply(aboutLinks, function(about_link){
  about_link$getElementAttribute('href')
})

edited Mar 22 '19 at 19:08

answered Mar 22 '19 at 17:24

supputuri

13,644
2
21
39

Still get that confusing `Error: object of type 'closure' is not subsettable` error, unfortunately. Must be something to do with the following `sapply` function? As in, your elements are definitely correct, but something after that isn't working – papelr Mar 22 '19 at 17:31
`link2 <- as.list(link2)` why you are doing this? – supputuri Mar 22 '19 at 17:36
I read somewhere that that could have been giving me that error.. but who knows. Same error either way – papelr Mar 22 '19 at 17:38
Done - same error. Might be my machine? Though, if I comment out the `sapply` part, and `print(link2)`, it does print out the correct number of links. Just the `sapply` throws that subsettable error – papelr Mar 22 '19 at 18:06
Ok, figured out the issue. We missed `s' while getting the `About this book` links. we should pass elements to `sapply`, not an element. – supputuri Mar 22 '19 at 19:13

RSelenium: clicking a link within another link

1 Answers1