Stack Overflow
Stack Overflow

Questions tagged [splash-js-render]

Splash JS is a javascript rendering service. It’s a lightweight web browser with an HTTP API, implemented in Python using Twisted and QT. It's Selenium's competitor.

https://splash.readthedocs.io/en/stable/

Splash - A javascript rendering service

Splash is a javascript rendering service. It’s a lightweight web browser with an HTTP API, implemented in Python using Twisted and QT. The (twisted) QT reactor is used to make the sever fully asynchronous allowing to take advantage of webkit concurrency via QT main loop. Some of Splash features:

  • process multiple webpages in parallel;
  • get HTML results and/or take screenshots;
  • turn OFF images or use Adblock Plus rules to make rendering faster;
  • execute custom JavaScript in page context;
  • write Lua browsing scripts;
  • develop Splash Lua scripts in Splash-Jupyter Notebooks.
  • get detailed rendering info in HAR format.
138 questions
1
vote
0 answers

Scrapinghub/Splash - Aquarium is not working on docker-compose

We are trying to use Aquarium, to set up the Scrapinghub/Splash. While installation, when I use "docker-compose up" to start the Splash, It throws the exception Traceback (most recent call last): File "/usr/local/bin/docker-compose", line 11,…
Mideen abdul gaffoor
  • 81
  • 1
  • 7
1
vote
0 answers

Scrapy Splash got detected by Trivago

When I use Splash to scrape Trivago, I got Captcha. It doesn't happen if I use curl or using a normal Scrapy request. Is there a way to use Splash but not get detected as a bot by Trivago?
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
1
vote
1 answer

scrapy-splash rendering more than the first page

I'm trying to scrape a website but need to use splash in all pages because their content created dynamically. right now it renders only the first page, but not the content page or the pagination pages. here's the code: import scrapy from…
dor272
  • 550
  • 1
  • 6
  • 26
1
vote
2 answers

Trying out Scrapy + Splash

So I'm playing around with Scrapy & Splash and I'm running into some issues. I tried running my spiders, and kept getting HTTP 502 & 504 errors. Okay, so I tried to check out Splash in my browser. First I did "sudo docker run -p 8050:8050 -p…
Craig
  • 161
  • 7
1
vote
2 answers

Splash server with Docker not running on Mac

I am trying to configure a Splash server with Docker in order to render javascript for scrapy. I downloaded and installed Docker Toolbox (The latest version of Docker does not install on my Macbook pro 2009 due to lack of CPU MMU support). I runned…
jbp
  • 115
  • 1
  • 6
1
vote
2 answers

Splash won't render all contents of page

I am using Splash v2.3.2 and I am trying to render a page but it is not rendering everything. It won't render images or dynamically loaded content. I am using my http://localhost:8050/ with script: function main(splash) local url =…
Maciek Semik
  • 1,872
  • 23
  • 43
1
vote
1 answer

How to Use Splash (JS Rendering Service) with a Proxy

It's configured automatically in Scrapy, but not in Curl or normal request. In curl, we can do this without any proxy: http://:8050/render.html?url=http://www.example.com/?timeout=10&wait=0.5 How to do it with proxy? I tried…
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
0
votes
0 answers

Scrapy-Splash is not able to click search button or send Enter key

So I was learning to use Splash 3.5v for scraping Dynamic websites and Splash was successful in sending text splash:send_text() to the search bar of the website https://www.olx.com.pk/. But Splash was not able to click the search button. I have…
Muhammad Ahmad
  • 84
  • 6
0
votes
0 answers

Splash doesn't render any JavaScript elements on website

I'm using Splash in a docker server with DigitalOcean to render a website (https://www.coop.se/handla/varor/mejeri-agg/kvarg-cottage-cheese/naturell-kvarg/kvarg-naturell-7392672001403) so that it'll load the title and the rest of the page. I feel…
G_us
  • 13
  • 3
0
votes
0 answers

Splash Connection refused on target localhost URL

I am trying to render JS with Splash, it works with normal URLs like https://google.com, etc but it does not work for URLs having addresses like http://localhost:8002/report. Error: {"error": 502, "type": "RenderError", "description": "Error…
Muhammad Asif Saleem
  • 198
  • 1
  • 1
  • 11
0
votes
0 answers

Get "LUA_ERROR" "webkit203" when trying to render a page containing a video by Splash

I use Scrapy + Splash to scrape data from news sites. For one of them though (Farhikhtegan Daily), while I have managed to render text/image-only pages, but those pages containing a specific type of video cannot be rendered due to an error…
Roozbeh
  • 11
  • 2
  • 2
0
votes
0 answers

Scrapy Splash - Not Rendering the full content

I'm trying to scrape this site https://tucson.craigslist.org/search/acc?postedToday=1#search=1~list~0~0, When i try it in splash web console and try to give a wait time of 30 seconds, sometimes it renders the full page and sometimes it is not…
Aravindan
  • 9
  • 1
  • 3
0
votes
0 answers

Scrapy-splash not send cookies in other requests

I use cookies to access the website. Everything is fine, I'm logged in. But after I use lua script to click on element => js sends an ajax with some information. And I realized that ajax doesn't send cookies with it. Similar case, when logged in, I…
Phong Nguyễn
  • 25
  • 5
0
votes
0 answers

Why my splash:select always fail in this simple code (Splash)?

I want to select the "2nd-page" button at the end of this website using Splash code (Docker/Browser ver). I even used the "copy Selector" function of Chrome to get to the extract part, I also checked with F12 and CtrlF and found the extract…
Hoang Minh Quang FX15045
  • 733
  • 1
  • 4
  • 15
0
votes
1 answer

Python, Splash select XPATH and click button

I want to use Python and Splash to select the 'Ich stimme zu' button with XPATH and then click. I can't use the CSS class because it has a dynamic name. URL:…
6394693473540
  • 1
1 2 3
…
9 10