R - RSelenium, press Next button & change dropdown options to 25

Question

OP Warning: I am not very good at HTML.

I am trying to use RSelenium remote-drive (browserName='phantomjs') to scrape some links on a login required page. I was able to handle the login part but when I try to extract the links in the table, I cannot scrape all of them because;

1- Table has a limited view of 10-row displays. This can be altered using the dropdown option up to 25.

 Things I have tried: 

 option <- remDr$findElement(using = 'xpath', "//*/option[@value = '25']")
 option$clickElement()

 As a result I get the ERROR: Element is hidden

2- I cannot press the next button at the bottom of the table to see the links in the next page.

 Similarly, I think I was able to find all 4 buttons using findElement(). But when I run;

 buttons <- remDr$client$findElement("class name", "ag-paging-button")
 nextbutton <- buttons[[3]]
 nextbutton$click()
 nextbutton$clickElement()

Then run the link extraction by "href" I get the same 10 links as if nothing had happened.

My issue can be fixed by just getting the part 2 done, but I would appreciate if I could get an answer for the first one too.

Here is how selected chunks of my HTML source code looks like.

<div class="pxl-aggrid-pagesize">
   "Displaying: "
   
   <select id="pxl-ag-grid-pageSelect">
      <option value="10">10</option>
       <option value="15">15</option>
        <option value="20">20</option>
         <option value="25">25</option>
   </select>
</div>
    
    
    
    
<div class="ag-paging-panel ag-font-style">
   <span class="ag-paging-page-summary-panel">
      <button class="ag-paging-button" ref="btFirst" disabled type="button">First</button>
       <button class="ag-paging-button" ref="btPrevious" disabled type="button">Previous</button>
        "
                 Page "
        <span ref="lbCurrent">1</span>
        " of "
        <span ref="lbTotal">0</span>
        <button class="ag-paging-button" ref="btNext" disabled type="button">Next</button>
         <button class="ag-paging-button" ref="btLast" disabled type="button">Last</button>

score 0 · Accepted Answer · answered Sep 18 '17 at 13:00

I was able to do this with mouseMoveToLocation() function. Surely, it would need modification based on the layout used by the website itself. However, for any good it might to; code is attached below.

  allButtons <- remDr$client$findElements("xpath", "//button[@class='ag-paging-button']")
    allButtonsText <- sapply(allButtons, function(x)x$getElementText())
    nextButtonNumber <- grep('Next', unlist(allButtonsText))    

  remDr$client$mouseMoveToLocation(webElement=allButtons[[nextButtonNumber]])
    remDr$client$click(1)

R - RSelenium, press Next button & change dropdown options to 25

1 Answers1