unknown command: crawl error

Question

I am a newbie to python. I am running python 2.7.3 version 32 bit on 64 bit OS. (I tried 64 bit but it didn't workout).

I followed the tutorial and installed scrapy on my machine. I have created one project, demoz. But when I enter scrapy crawl demoz it shows an error. I came across this thing when i hit scrapy command under (C:\python27\scripts) it shows:

C:\Python27\Scripts>scrapy
Scrapy 0.14.2 - no active project

Usage:
  scrapy <command> [options] [args]

Available commands:
  fetch         Fetch a URL using the Scrapy downloader
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

Use "scrapy <command> -h" to see more info about a command

C:\Python27\Scripts>

I guess their is something missing in installation can anybody help please .. Thanks in advance..

What is the exact error you are getting? The help response for scrapy isn't going to help narrow down the issue. — Drahkar, Apr 12 '12 at 12:06

score 83 · Answer 1 · edited Jun 20 '20 at 09:12

83

You should run scrapy crawl spider_name command being in a scrapy project folder, where scrapy.cfg file resides.

From the docs:

Crawling

To put our spider to work, go to the project’s top level directory and run:

scrapy crawl dmoz

edited Jun 20 '20 at 09:12

Community

1
1

answered Apr 12 '12 at 12:30

warvariuc

57,116
41
173
227

1

it works fine. it is important to run crawl command in a folder contains cfg file – Mohammad Heydari Nov 17 '18 at 18:58

Subin Shrestha · Answer 2 · 2018-01-21T04:29:27.027

12

You can run scrapy crawl demoz code from your scrapy project folder which you have created using following command

scrapy startproject tutorials

For example, if you have started scrapy project of name tutorials, then go to tutorials folder first and run crawl command from there

scrapy crawl demoz

edited Jan 21 '18 at 04:29

answered May 20 '16 at 08:18

Subin Shrestha

1,212
11
11

score 0 · Answer 3 · answered Mar 17 '20 at 04:07

0

If you have already an existing project and suddenly this error "Scrapy 1.3.3 - no active project" came, in my case when I open my project, I cant see the scrapy.cfg file. What I did was, I just made some little changes on my project like removing some extra line and deployed it from github to instance and it worked!

answered Mar 17 '20 at 04:07

bonifacio_kid

633
5
8

Can you provide a `diff` of your changes, so others can see what you did? TIA. – benc Mar 17 '20 at 04:54

unknown command: crawl error

3 Answers3

Linked