Custom JSON Response from Scrapy Spider Deployed via Scrapyd

Question

I need to find a way to make my Scrapy spider return a custom JSON response. It is deployed via scrapyd using schedule.json. Schedule.json responds with JobID and Status, but I'd like to add some more data to that response.

If there's a way I could do it by creating a separate response and then make a separate request that would work as well.

Much appreciated!

score 1 · Accepted Answer · answered Dec 06 '14 at 00:48

1

enter image description here

^{(taken from Learn to Read the Source, Luke).}

It is doable.

Implement your own resource based on scrapyd.webservice.WsResource, write a render_POST() method. Then, tell scrapyd to use your schedule.json resource implementation:

[services]
schedule.json = project.module.MySchedule

answered Dec 06 '14 at 00:48

alecxe

462,703
120
1,088
1,195

Sorry for the image, just love to quote it :) – alecxe Dec 06 '14 at 00:50
Thanks! What docs? I've read through the Scrapyd documentation and didn't come across that... – ChristianTL Dec 09 '14 at 23:52

Custom JSON Response from Scrapy Spider Deployed via Scrapyd

1 Answers1