0

I am writing a scraping app which is running on Lambda. I want to combine Scrapy with Splash, but I am not sure that can Splash run on lambda.

Nimantha
  • 6,405
  • 6
  • 28
  • 69
Nhan Phan
  • 1,262
  • 1
  • 14
  • 32

2 Answers2

1

It depends on how you are trying to install Splash.

  1. If you want to run it using docker image, then take a look at Elastic container service. Lambda is not ideal for running containerized app.
  2. If you are going to run it by manual way, like mentioned here you can do so. But as per the doc, its not recommended. As AWS Lambda supports Python, this should not be problem, as long as it doesn't require any other dependency. Also consider the starting time for your Splash server. Lambda allocates instances on the fly, So it might take time to start your application (AWS lambda cold start).
Sangam Belose
  • 4,262
  • 8
  • 26
  • 48
  • 1
    I think this answer is outdated, AWS Lambda now supports Docker images: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/, https://docs.aws.amazon.com/lambda/latest/dg/images-create.html Not sure what "not ideal" refers to exactly, and if that concern is still applicable. – Carsten Jul 22 '22 at 08:21
0
  • I think its worth having a look at lambda runtime API and lambda layers. aws docs
amittn
  • 2,249
  • 1
  • 12
  • 19