0

I'm trying to get a website through Scrapy in the shell, 

$ scrapy shell -s NAME="Mozilla/5.0" "http://www.yapo.cl/chile/inmuebles?ca=15_s&l=0&cmn=&st=a"

2017-08-21 20:55:07 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying <GET http://www.yapo.cl/chile/inmuebles?ca=15_s&l=0&cmn=&st=a> (failed 3 times): 504 Gateway Time-out

but Scrapy through me a 504 error which I cannot figure out, any guess what could it be?.

Raúl Bernardo
  • 1

2 Answers2

1

You are probably trying to set user agent string on command line, but using wrong setting (NAME). Try it with:

$ scrapy shell -s USER_AGENT="Mozilla/5.0" "http://www.yapo.cl/chile/inmuebles?ca=15_s&l=0&cmn=&st=a"

Like that, I get:

2017-08-22 07:40:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://www.yapo.cl/chile/inmuebles?ca=15_s&l=0&cmn=&st=a> (referer: None)
Tomáš Linhart
  • 9,832
  • 1
  • 27
  • 39
0

You were banned or something like that. Try using another ip address. On my computer it gives this:

2017-08-22 00:07:43 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://www.yapo.cl/chile/inmuebles?ca=15_s&l=0&cmn=&st=a> (referer: None) ['partial']
Hebmaster
  • 27
  • 8