1

I am new to scraping and have successfully scraped tables from these websites:-

https://www.numberfire.com/nba/daily-fantasy/daily-basketball-projections/guards
https://www.dailyfantasyfuel.com/nba/projections/draftkings/
https://www.sportsline.com/nba/expert-projections/simulation/

But this website:-

https://www.lineups.com/nba/nba-fantasy-basketball-projections

seems very tricky.

1. I have tried to read it as JSON

r <- read_html('https://www.lineups.com/nba/nba-fantasy-basketball-projections/') %>% html_element('script#__NEXT_DATA__') %>% html_text() %>% jsonlite::parse_json()

2. from rvest methods

    data <- "https://www.lineups.com/nba/nba-fantasy-basketball-projections" %>%
      read_html %>%
      html_nodes('script') %>%
      html_text()

3. As well as RSelenium but with no success.

Could you kindly tell me how to deal with these kinds of Tables found at "www.lineups.com" ?

Thanks

Nad Pat
  • 3,129
  • 3
  • 10
  • 20
Abdul Basit Khan
  • 646
  • 1
  • 6
  • 19
  • Can you download the csvs instead? The site's license specifically requests no scraping and it seems even regular highlighting/ctrl+c is not possible. https://www.lineups.com/license It's possible to script rvest/selenium to trigger the download, and then any typical csv function can be used to read the data in. – Jul Mar 05 '22 at 10:19
  • I see. But I am planning to incorporate it into Shiny App and post it on Web (Shiny Server). Thus, exporting is as csv and then importing it into R Workspace seems like a complicated task as well. – Abdul Basit Khan Mar 05 '22 at 10:27

1 Answers1

2

Using RSelenium

library(RSelenium)
library(rvest)
library(dplyr)
driver = rsDriver(browser = c("firefox"))

remDr <- driver[["client"]]

url <- 'https://www.lineups.com/nba/nba-fantasy-basketball-projections'
remDr$navigate(url)

#get all the tables from webapage 
df = remDr$getPageSource()[[1]] %>% 
    read_html() %>% html_table()
[[2]]
# A tibble: 51 x 31
   Player   Player Player Player Player Player Player Player ``    ``    ``    ``    ``    ``    Game  Game  Game  Game  Game  `Projected Game~ `Projected Game~
   <chr>    <chr>  <chr>  <chr>  <chr>  <chr>  <chr>  <chr>  <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>            <chr>           
 1 "Name"   Team   Pos    Proje~ Salary Pts/$~ FPPM   USG%   Pos   Proj~ Sala~ Pts/~ FPPM  USG%  Opp   DvP   Spre~ Total O/U   MINS             PTS             
 2 "Nikola~ DEN    C      58.37  $11,0~ 5.3    1.8    31.3%  HOU   30    13.5  126   238   33    27.7  7.7   13.7  1.1   0.9   5                18.8            
 3 "Gianni~ MIL    PF     57.84  $11,3~ 5.1    1.8    35.1%  CHI   27    -5    122.~ 240   33    30.5  5.9   11.7  1     1.4   7.8              19.1            
 4 "Joel E~ PHI    C      53.35  $10,8~ 4.9    1.7    37.4%  CLE   5     7     112   216.5 32    29.4  4.2   11    0.9   1.4   8.9              19
Nad Pat
  • 3,129
  • 3
  • 10
  • 20
  • This is an awesome solution. Have been struggling for this for days. Thanks for such a simple and elegant solution. @Nad Pat, I have run into another problem similar to this. I would be obliged if you could view it as well. Thanks – Abdul Basit Khan Mar 05 '22 at 15:09