I have deployed some jobs on a couple of machines with crontab. Recently, I have two problems. 1.Some problems happened on one of these machines:jobs are not running and staying in pending queue for a lot of days. 2.Unlike problem above, some jobs on another machine are staying with running status for more than 1 day. These jobs should finish in about less than one hour and their logs shows that they have finished already (most log files have been removed and a couple of log files of these jobs exist with content of finish notice).
jobs info shows below:
Project Spider Job PID Runtime Log Items
Pending
weibo_spider weibo_spider b2fc68949b1311e493ed0025908e45a7
ifeng_news wangyi_spider 0b449fdeac3611e493ed0025908e45a7
ifeng_news qq_spider 0dda73f4ac3611e493ed0025908e45a7
ifeng_news infzm_spider 106cb1a4ac3611e493ed0025908e45a7
ifeng_news china_spider 11424670ac3611e493ed0025908e45a7
ifeng_news people_spider 13d38e80ac3611e493ed0025908e45a7
ifeng_news sohu_news_spider 15011d40ac3611e493ed0025908e45a7
ifeng_news sohu_news_spider 154488beac3611e493ed0025908e45a7
ifeng_news takungpao_spider 183df654ac3611e493ed0025908e45a7
ifeng_news people_spider 59b99b52ac5d11e493ed0025908e45a7
ifeng_news people_spider 5b437e94aeb911e493ed0025908e45a7
sns douban_spider b0cebbba9a3411e493ed0025908e45a7
sns zhihu_spider c7b866be9a3411e493ed0025908e45a7
sns douban_spider d227a9ca9a3411e493ed0025908e45a7
sns renren_spider df6482109a3511e493ed0025908e45a7
Running
Finished
weibo_spider weibo_spider a4225cb0998e11e493ed0025908e45a7 0:52:41.606361 Log Items
weibo_spider weibo_spider 0654c0b0999611e493ed0025908e45a7 0:00:12.500178 Log Items
On another machine:
Project Spider Job PID Runtime Log Items
Pending
Running
ifeng_news people_spider e791bf38b03411e49b32ac853d971155 40906 0:00:01.822573 Log Items
ifeng_news sina_spider ce8b4754b03211e49b32ac853d971155 47189 0:15:01.822328 Log Items
ifeng_news sina_spider ce939f26b03211e49b32ac853d971155 47280 0:14:56.822744 Log Items
ifeng_news ifeng_spider c8b5901cb03411e49b32ac853d971155 36879 0:00:46.823007 Log Items
ifeng_news ifeng_spider cecf2f76b03411e49b32ac853d971155 37129 0:00:41.822699 Log Items
ifeng_news qq_spider 5b1f66e4aea011e49b32ac853d971155 54869 2 days, 0:15:41.822598 Log Items
sns renren_spider 6e29baeaaf6911e49b32ac853d971155 35236 1 day, 0:16:31.822673 Log Items
weibo_spider weibo_spider f90fce66aecd11e49b32ac853d971155 14109 1 day, 18:49:21.822745 Log Items
ifeng_news qq_spider 04e0616aaea111e49b32ac853d971155 21047 2 days, 0:11:11.822737 Log Items
ifeng_news qq_spider b67686daaea011e49b32ac853d971155 12836 2 days, 0:13:11.822703 Log Items
sns renren_spider 945857f4af6811e49b32ac853d971155 33898 1 day, 0:21:06.822709 Log Items
Finished
ifeng_news huanqiu_spider 309e0e90b03311e49b32ac853d971155 0:00:03.105042 Log Items
ifeng_news ifeng_spider fd4d8502b03211e49b32ac853d971155 0:01:24.509558 Log Items
ifeng_news huanqiu_spider 30aee9e0b03311e49b32ac853d971155 0:00:02.583991 Log Items
the log of job 6e29baeaaf6911e49b32ac853d971155 like things below:
2015-02-08 16:07:02+0800 [renren_spider] DEBUG: Crawled (200) <GET http://browse.renren.com/s/all/ajax?q=%E9%B8%9F%E5%B7%A2&limit=100&p=&offset=600&sort=1&range=0&t=2&defaultFirst=0> (referer: http://browse.renren.com/s/all/ajax?q=%E9%B8%9F%E5%B7%A2&limit=100&p=&offset=500&sort=1&range=0&t=2&defaultFirst=0)
2015-02-08 16:07:02+0800 [renren_spider] INFO: Closing spider (finished)
2015-02-08 16:07:02+0800 [renren_spider] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 899049,
'downloader/request_count': 36,
'downloader/request_method_count/GET': 23,
'downloader/request_method_count/POST': 13,
'downloader/response_bytes': 371895,
'downloader/response_count': 36,
'downloader/response_status_count/200': 23,
'downloader/response_status_count/404': 13,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2015, 2, 8, 8, 7, 2, 736353),
'log_count/DEBUG': 38,
'log_count/INFO': 7,
'request_depth_max': 6,
'response_received_count': 36,
'scheduler/dequeued': 36,
'scheduler/dequeued/memory': 36,
'scheduler/enqueued': 36,
'scheduler/enqueued/memory': 36,
'start_time': datetime.datetime(2015, 2, 8, 8, 6, 54, 581699)}
2015-02-08 16:07:02+0800 [renren_spider] INFO: Spider closed (finished)
