2

  1. I would like to start scrapyd as service but when I start scrapyd, if I close the SSH session the service scrapyd close automatically.

  2. When I try to start as service like this I have an error :

    root@vps:~# service scrapyd start
scrapyd: Failed to start scrapyd.service: Unit scrapyd.service failed to load: No such file or directory.

  3. And when i try to start daemon scrapyd the CURL request return :

    {"status": "error", "message": "Use \"scrapy\" to see available commands", "node_name": "vps"}

Can someone help me to start my scrapyd as service please !

Pixel
  • 900
  • 1
  • 13
  • 31
  • Here you can take a look at this answer https://stackoverflow.com/a/51319103/5957834 . It is good work around. – Nandesh Jul 13 '18 at 06:46

2 Answers2

0
daemon --chdir=/home/Crawler scrapyd

I need to set the --chdir to load the service on the Scrapy folder !

Pixel
  • 900
  • 1
  • 13
  • 31
0

I could not get my scrapyd start through /etc/rc.local or crontab so I found a work around. I am sure there will be a better way but for mean time this worked for me.

I created a python file start.py.

import os

os.system('/usr/bin/python3 /home/ubuntu/.local/bin/scrapyd > /home/ubuntu/scrapy.log &')

And simply call the python file start.py through crontab.

Add this to crintab file by cronat -e.

@reboot /usr/bin/python3 /home/ubuntu/start.py

Basically what I did is execute the scrapy through shell by calling the python filr through crontab.

If anyone finds something better please comment.

Nandesh
  • 4,437
  • 2
  • 20
  • 26