0

In my settings.py I have:

ITEM_PIPELINES = {
    'turing.pipelines.InitFieldsNotInitializedPipeline': 299,
    'turing.pipelines.SetNoneIfFieldEmptyPipeline': 300,
    'turing.pipelines.CheckCategoryPipeline': 301,
    'turing.pipelines.CheckContactPipeline': 302,
}

And it works great. But sometime I want run the spider without ANY pipelines. When I run

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s ITEM_PIPELINES=[] example_spider

I get this error:

return d.iteritems(**kw)
exceptions.AttributeError: 'str' object has no attribute 'iteritems'

How can I run the spider without the pipelines?

So far I tried:

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s ITEM_PIPELINES=[] example_spider

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s ITEM_PIPELINES={} example_spider

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s "ITEM_PIPELINES=[]" example_spider

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s "ITEM_PIPELINES={}" example_spider

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s ITEM_PIPELINES=['turing.pipelines.InitFieldsNotInitializedPipeline': 299,] example_spider

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s ITEM_PIPELINES={'turing.pipelines.InitFieldsNotInitializedPipeline': 299,} example_spider

Others combinations Look in the docs http://doc.scrapy.org/en/latest/topics/settings.html

Hopefully you can help me. Thanks.

Luis G. Angel
  • 31
  • 3
  • I don't think there is a way to currently do that, as every argument is being parsed as a string in scrapy, anyway it is worth asking it as a feature [here](https://github.com/scrapy/scrapy/issues) – eLRuLL May 27 '16 at 16:15
  • There isn't. I post the issue in scrapy's github. https://github.com/scrapy/scrapy/issues/2018#issuecomment-222182862 – Luis G. Angel Jun 09 '16 at 22:05

1 Answers1

1

I answered on github, but I'm putting it here as well:

You need to escape the {} like so: \{\}

scrapy crawl -s FEED_URI=stdout: -s FEED_FORMAT=json -s ITEM_PIPELINES=\{\} -a test_extract_url=http://example.com/ -L ERROR c_example

Sam Texas
  • 1,245
  • 14
  • 30