Scrapyd pass parameters when deploying

Question

This is a simple example of a scrapy.cfg file:

[settings]
default = crawly.settings

[deploy:s1]
url = http://localhost:6800
project = my_project

I wanna know if I could pass any parameters to my scrapyd instance using this file. What I wanna do is to pass a parameter to one of my middlewares. You can pass parameters to your spider or override your settings but not anything with middlewares. I was thinking if a parameter like project could be passed to myserver, I could use that in my middleware.

Downvote without a comment!? nice. – AliBZ Jan 01 '14 at 01:34 — AliBZ, Jan 01 '14 at 01:34
how is it better than using settings.py? – Guy Gavriely Jan 01 '14 at 03:08 — Guy Gavriely, Jan 01 '14 at 03:08
I need that argument to be different for different servers. – AliBZ Jan 02 '14 at 17:24 — AliBZ, Jan 02 '14 at 17:24

score 0 · Answer 1 · answered Jan 02 '14 at 19:00

0

check out this corresponding on scrapy github, check the code if they ended up integrating the suggested change or take pablo advice for post egg generation

answered Jan 02 '14 at 19:00

Guy Gavriely

11,228
6
27
42

Scrapyd pass parameters when deploying

1 Answers1