getting spider instance from scrapyd

Question

Is there a way to get the instance of the spider that runs when you schedule a run using scrapyd? I need to access attributes in the spider to handle outside the run and can't use a json/csv file to do this.

I found what I needed in [link](http://stackoverflow.com/questions/4113275/scrapy-pipeline-spider-opened-and-spider-closed-not-being-called) : using the spider right before the spider closes. Can anyone close this? — Jean Ventura, Nov 20 '13 at 10:59

score 0 · Accepted Answer · edited May 23 '17 at 11:56

0

I found what I needed in here: using the spider_close method to run the code I need right before the spider closes.

You need to add the following to the pipeline's __init__ method (otherwise it never receives the spider_closed signal):

dispatcher.connect(self.spider_opened, signals.spider_opened)
dispatcher.connect(self.spider_closed, signals.spider_closed)

edited May 23 '17 at 11:56

Community

1
1

answered Dec 26 '13 at 13:04

Jean Ventura

27
10

getting spider instance from scrapyd

1 Answers1