Scrapyd Permission Denied on Deploy

Question

I'm very new to Scrapyd, and am trying to deploy. I am running on Ubuntu 12.04 and installed the ubuntu version of Scrapyd. When I run scrapy deploy default -p pull_scrapers it returns

Packing version 1407616523
Deploying to project "pull_scrapers" in http://localhost:6800/addversion.json
Server response (200):
{"status": "error", "message": "[Errno 13] Permission denied:'/var/lib/scrapyd/eggs/pull_scrapers'"}


Any idea why this would be? I looks to be a credentials problem, but so far as I know it doesn't have any. Any help is appreciated.

EDIT:

After turning debug=on in the conf.d/000-default file, the error is

root@robert-GA-870A-UD3:/home/feed/Desktop/feed/scrapers# scrapy deploy default
Packing version 1407875943
Deploying to project "pull" in http://localhost:6800/addversion.json
Server response (200):
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/scrapyd/webservice.py", line 18, in render
    return JsonResource.render(self, txrequest)
  File "/usr/lib/pymodules/python2.7/scrapy/utils/txweb.py", line 10, in render
    r = resource.Resource.render(self, txrequest)
  File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 216, in     render
    return m(request)
  File "/usr/lib/pymodules/python2.7/scrapyd/webservice.py", line 65, in render_POST
    self.root.eggstorage.put(eggf, project, version)
  File "/usr/lib/pymodules/python2.7/scrapyd/eggstorage.py", line 21, in put
    makedirs(eggdir)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/lib/scrapyd/eggs/pull'

add `debug = on` to `scrapyd.conf`, that will show the full error traceback — Padraic Cunningham, Aug 09 '14 at 20:50
I did. do you know what it means? it's not very readable from my knowledge standpoint — robert, Aug 09 '14 at 21:32
I don't have a scrapyd.conf. I do have ../scrapyd/conf.d/000-Default though and that is the output of it. — robert, Aug 10 '14 at 00:30
it is strange that there is no info regarding a permission error in the debug output.how did you install scrapy? — Padraic Cunningham, Aug 10 '14 at 00:35
I install scrapy following the ubuntu recommended way. http://doc.scrapy.org/en/latest/topics/ubuntu.html#topics-ubuntu — robert, Aug 10 '14 at 03:37

score 3 · Answer 1 · answered Aug 11 '14 at 11:34

3

This error can be somewhat misleading. Try running from inside your project (the dir where setup.py is located):

python setup.py clean -a bdist_egg -d /tmp/scrapydeploy-8NnURF

This will essentially do the same thing that scrapy deploy does, but it'll show a lot more output (you can change the name of the dir in tmp, this was just for example, scrapy generates something random after the '-').

In my case there was a problem with the permissions of the build directory in my project (scrapy deploy failed to delete the old files inside), so I just deleted the directory, as it's generated automatically anyway.

PS: I would have left a comment, but I don't have enough reputation.

answered Aug 11 '14 at 11:34

todinov

504
3
11

1

Thanks toninov. I reinstalled scrapyd because I couldn't get that way to work. I've also updated my post with the most recent error. (so many errors haha) – robert Aug 12 '14 at 20:40
Can you post the output of the command I posted? It might give a bit more info about what is actually happening. When you reinstalled, did you remove scrapyd with purge to remove all it's configuration or did you do a normal remove? – todinov Aug 13 '14 at 05:09

Scrapyd Permission Denied on Deploy

1 Answers1