crawler4j always returns fatal transport error

Question

This is what I get for any seed I add to crawler4j.

ERROR [Crawler 1] Fatal transport error: Connection to http://example.com refused while fetching http://example.com/page.html (link found in doc #0)

This is really weird for me. I don't know what causes it.

What happens when you visit `http://example.com/page.html` in a browser from the same system? — Jim Garrison, May 10 '13 at 03:44

score 3 · Accepted Answer · answered Jun 30 '13 at 15:45

3

try to increase the politeness setting of the crawler. It always happens to me when the server that I want to crawler is not good.

answered Jun 30 '13 at 15:45

Peter Duan

1 Answers1