1

I have a flask sever running on localhost:5000 and a scrapyd running on localhost:6800 with only one spider. My problem is that i have scraped some data and i want to send the data to the flask server for some processing over the extracted data.
If the data had to be send to some database, say MongoDB then we could have used pymongo.Connection in item pipeline. Is there any similar approach for sending the data to flask server ?

Vivek Anand
  • 621
  • 1
  • 7
  • 15
  • May be you should try to send http-request to your flask server? – ikoverdyaev Aug 27 '15 at 05:33
  • I tried this, but it gives another problem. We have to return some value from the flask route otherwise it raises ValueError. But, here i don't want to receive anything back to scrapyd – Vivek Anand Aug 27 '15 at 08:49
  • If there is solution for that ValueError, then it would be great. – Vivek Anand Aug 27 '15 at 08:56
  • Did you read this http://stackoverflow.com/questions/25034123/flask-value-error-view-function-did-not-return-a-response? – ikoverdyaev Aug 27 '15 at 09:11
  • Well, that value error is not coming now. I just returned any random string. But, there is still something that i can't understand. I am getting all the data but, the following i can't understand. First it gives 200 then 400 why? 127.0.0.1 - - [27/Aug/2015 14:33:08] "POST /reviews HTTP/1.1" 200 - 127.0.0.1 - - [27/Aug/2015 14:33:08] "POST /reviews HTTP/1.1" 400 - Is it because i am returning a random string ? – Vivek Anand Aug 27 '15 at 09:12
  • Leave it. It's fine now. I had made two requests at different positions. One of them was wrong. – Vivek Anand Aug 27 '15 at 09:15

0 Answers0