Stack Overflow
Stack Overflow

Questions tagged [splash-js-render]

Splash JS is a javascript rendering service. It’s a lightweight web browser with an HTTP API, implemented in Python using Twisted and QT. It's Selenium's competitor.

https://splash.readthedocs.io/en/stable/

Splash - A javascript rendering service

Splash is a javascript rendering service. It’s a lightweight web browser with an HTTP API, implemented in Python using Twisted and QT. The (twisted) QT reactor is used to make the sever fully asynchronous allowing to take advantage of webkit concurrency via QT main loop. Some of Splash features:

  • process multiple webpages in parallel;
  • get HTML results and/or take screenshots;
  • turn OFF images or use Adblock Plus rules to make rendering faster;
  • execute custom JavaScript in page context;
  • write Lua browsing scripts;
  • develop Splash Lua scripts in Splash-Jupyter Notebooks.
  • get detailed rendering info in HAR format.
138 questions
0
votes
2 answers

Splash does not connect to proxy using any of the 3 ways described in documentation

Splash browser does not send anything to through the http proxy. The pages are fetched even when the proxy is not running. I am using scrapy with splash in python 3 to fetch pages after authentication for a an Angular.js website. The script is able…
neoinageo
  • 31
  • 5
0
votes
1 answer

How to use splash:mouse_press in Scrapy-Splash

I am trying yo click the display button on a website with no success and I really had no idea how to do it but I came across something that might work, splash:mouse_press. Will that work with scrapy-splash and if so how do I implement it? import…
Tim
  • 191
  • 2
  • 28
0
votes
2 answers

Can we deploy Splash on Aws Lambda?

I am writing a scraping app which is running on Lambda. I want to combine Scrapy with Splash, but I am not sure that can Splash run on lambda.
Nhan Phan
  • 1,262
  • 1
  • 14
  • 32
0
votes
1 answer

How to run splash using docker toolbox

I am trying out scrapy with splash to scrape dynamic content off the web, I'm on a windows 10 Home Edition. Is there an way to use Docker tool box instead of docker-desktop so as to work with splash? The docker toolbox says, it is an alternative for…
ram
  • 23
  • 6
0
votes
0 answers

scrapy splash - SplashFormRequest with wrong args

I'm new to splash and I just implemented it into scrapy and tried to use it to replace one classic scrapy request with it (as my pages I want to scrape uses some js). so far I have : for a in res_json['data']: req =…
Ayra
  • 328
  • 2
  • 12
0
votes
1 answer

Is there anyway to stop a Splash script while executing?

Suppose you execute a large script on Splash which needs to perform 5 minutes worth of operations. With the correct timeout parameter this is completely possible. Suppose that when splash starts processing the script you want to prematurely stop…
LaserJesus
  • 8,230
  • 7
  • 47
  • 65
0
votes
1 answer

Scrapy Splash + Crawlera in Linux always get 503 service unavailable error

When I use Scrapy Splash + Crawlera in my Linux server, it always gets 503 errors. It works just fine in Windows. Why is that?
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
0
votes
1 answer

Blank spaces between scraped values in Scrapy

I'm trying to use Scrapy to scrape some objects from the following page: https://www.reclameaqui.com.br/indices/lista_reclamacoes/?id=9980&page=1&size=10&status=ALL Using the following code: class MySpider(scrapy.Spider): name = 'reclame_aqui' …
Gabriel Naslaniec
  • 15
  • 3
0
votes
1 answer

Scrapy splash can't find element

Problem: I am using scrapy splash to scrape a web page. However it seems the css path for imageURL does not return any element but the ones for name and category worked fine. (xpath and selector are all copied directly from Chrome.) Things I've…
Tinyik
  • 457
  • 6
  • 21
0
votes
1 answer

Splash UI doesn't start on windows

I've setted splash on docker with Docker Toolbox & Docker Quickstart Terminal. It gives this output: $ docker run -p 8050:8050 scrapinghub/splash 2018-02-27 14:29:21+0000 [-] Log opened. 2018-02-27 14:29:21.129146 [-] Splash version: 3.2 2018-02-27…
Lore
  • 1,286
  • 1
  • 22
  • 57
0
votes
0 answers

R 'Splashr' - error on Windows

I'm trying to get the R package "Splashr" working following this tutorial. Ive installed Docker for windows successfully, Docker SDK for Python and (hopefully) the dependent Python packages. I've set the path for Python in System Variables and…
symeboy
  • 31
  • 5
0
votes
0 answers

send python get request with splash and custom headers

I want to use Python requests with splash browser (https://splash.readthedocs.io/en/stable/) and custom headers to crawl some data from a website. However, before starting the crawling itself I decided to check on this website…
Ostap Didenko
  • 446
  • 2
  • 6
  • 19
0
votes
1 answer

Splash stops with error "unregister_netdevice: waiting for lo to become free"

kernel:[1476369.644116] unregister_netdevice: waiting for lo to become free. Usage count = 1 I got that error then Splash will stop working and I have to run the docker from the beginning again. I heard it's related with docker. How to stop getting…
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
0
votes
2 answers

Scrapy Splash on Ubuntu server: got an unexpected keyword argument 'encoding'

The Scrapy Splash I am using is working just fine on my local machine, but it returns this error when I use it on my Ubuntu server. Why is that? Is it caused by low memory? File "/usr/local/lib64/python2.7/site-packages/twisted/internet/defer.py",…
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
1 2 3
…
9
10