Scrapy: how to disable or change log?

Question

I've followed the official tutoral of Scrapy, it's wonderful!

I'd like to remove all of DEBUG messages from console output. Is there a way?

2013-06-08 14:51:48+0000 [scrapy] DEBUG: Telnet console listening on 0.0.0.0:6029
2013-06-08 14:51:48+0000 [scrapy] DEBUG: Web service listening on 0.0.0.0:6086

The doc told about to set a LOG_LEVEL, but ... in which file ?

Replying, please make referring to this directory structure. It's the mine. Plus, I've a 'test.py' in spyder folder

tutorial/
    scrapy.cfg
    tutorial/
        __init__.py
        items.py
        pipelines.py
        settings.py
        spiders/
            __init__.py
            ...

Where. in which file, and HOW must i set the log_level ?

score 55 · Accepted Answer · edited Feb 06 '17 at 16:14

55

You need to add the following to your settings.py document:

LOG_LEVEL = 'INFO'

See the documentation.

edited Feb 06 '17 at 16:14

Maximilian Kohl

632
1
9
23

answered Jun 08 '13 at 15:01

icecrime

74,451
13
99
111

4

thanks ! I read the link, but I was not founding where the doc explain in which file to the settings ! – realtebo Jun 08 '13 at 15:03
1

@realtebo file: `tutorial/tutorial/settings.py` – Mykola Vasilaki Jan 24 '19 at 10:47
1

But it still shows information about scrappy, I want only to output logs what I specifically output in a code. – holms Feb 07 '20 at 19:12
2

So change 'INFO' to 'WARNING', 'ERROR', or 'CRITICAL' – Andy Thompson Oct 17 '22 at 07:43

Capi Etheriel · Answer 2 · 2015-09-17T18:41:15.810

28

You can also disable it completely with LOG_ENABLED=False. And you can pass settings when invoking the command: scrapy crawl myspider -s LOG_ENABLED=False

edited Sep 17 '15 at 18:41

answered Jun 10 '13 at 23:32

Capi Etheriel

3,542
28
49

Does case really matter for the `FALSE/False` values? – Wolf Jul 23 '15 at 11:37
1

I use the above command and get the result: "ValueError: invalid literal for int() with base 10: 'FALSE'". I try to replace "FALSE" with "0" and no error again now. – realjin Sep 17 '15 at 09:24
@Wolf it seems False will be parsed as boolean, FALSE will be parsed as string :/ – Capi Etheriel Sep 17 '15 at 18:42

score 11 · Answer 3 · answered Nov 18 '20 at 14:37

11

You can also use --nolog in command prompt.

> scrapy crawl myspider --nolog

--nolog command Sets LOG_ENABLED to False

see documentation scrapy command line options

answered Nov 18 '20 at 14:37

faizan-abdul-haq

111
1
5

Scrapy: how to disable or change log?

3 Answers3