1

I have a crawler running since a few days. I want to pause the crawler in order to do something else on the system. Scrapy documentation says, this can be done using telnet console but I am unable to login into telnet console. Here are the processes running in the system:

[root@xxx tmp]# telnet localhost 6073
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
[root@xxx tmp]# ps aux | grep scrapy 
root      5504  0.0  0.0 110400   860 pts/1    S+   04:31   0:00 grep scrapy
root     31457  4.0  1.9 774880 299436 pts/1   Sl   Sep21 141:27 /usr/local/pyenv/bin/python2.7 /usr/local/pyenv/bin/scrapy crawl myCrawler

Any help is appreciated. Thanks.

Andrés Pérez-Albela H.
  • 4,003
  • 1
  • 18
  • 29
Mayank Jaiswal
  • 12,338
  • 7
  • 39
  • 41

1 Answers1

1

Hah, here I am answering my own question. As stated in documentation, scrapy runs on port range [6023, 6073]. So to find the port being used, I used this command:

netstat -l

Output:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
    tcp        0      0 xxx.localdomain:6025      *:*                         LISTEN      
    tcp        0      0 *:27017                     *:*                         LISTEN      
    tcp        0      0 *:mysql                     *:*                         LISTEN      
    tcp        0      0 *:6379                      *:*                         LISTEN      
    tcp        0      0 *:webcache                  *:*                         LISTEN

"6025" port is what I was looking for.

Mayank Jaiswal
  • 12,338
  • 7
  • 39
  • 41
  • As stated on the docs (http://doc.scrapy.org/en/1.0/topics/telnetconsole.html#telnetconsole-port) TELNETCONSOLE_PORT is a port range variable. That said, it could choose any port in this range [6023, 6073]. Please correct your answer so other people don't get confused. – Andrés Pérez-Albela H. Sep 24 '15 at 06:24
  • Hi @AndrésPérez-AlbelaH. I made the language more straightforward. I hope its clear now. Feel free to revert back. – Mayank Jaiswal Sep 29 '15 at 08:47