2

I'm trying to run a scrapy spider on scrapinghub, and I want to pass in some data. I'm using their API to run the spider:

http://doc.scrapinghub.com/api/jobs.html#jobs-run-json

They have an option for job_settings, which seems relevant, but I can't figure out how to access the job_settings data in my Spider class. What is the correct approach here?

Sam Lee
  • 9,913
  • 15
  • 48
  • 56

1 Answers1

5

This job_settings shall be merged directly into the Scrapy settings, with a higher precedence (of 40, IIRC).

The Scrapy settings could be accessed via a .settings attribute of a spider instance, e.g. you could use self.settings if self is a scrapy.Spider instance.

starrify
  • 14,307
  • 5
  • 33
  • 50