13

There is a bot/spider crawling my websites very fast. The useragent is 'ltx71 - (http://ltx71.com/)' and it has serval ips:

52.3.127.144 and 52.3.105.23

On the website it says just this:

LTX71

We continuously scan the internet for security research purposes. Our crawling is not malicious and only notes summary information for a page.

If you have further questions please contact research@ltx71.com

I'm wondering what research purposes. I cannot find anything about it on Google. Should i block this bot?

Bo Pennings
  • 945
  • 1
  • 10
  • 20
  • I've seen it pop up, and it acts fairly well. Mostly it seems to do only fairly light crawling of regular content, and some sites is requested only the robots.txt file. At least for one site, where search result URLs had been disallowed, it still made a few request for those. But nothing that seemed worrisome. But if it's hammering your site(s), then I'd block it. If you're feeling nice you could try adding Crawl-delay lines if you haven't already, to see if it helps. – bjrn Aug 01 '16 at 15:14
  • @bjrn thanks good tip to slow it down in the crawl-delay. I've added it to the list :) – Bo Pennings Aug 02 '16 at 19:40
  • 1
    no more info about it? – LucScu Aug 08 '16 at 10:12
  • 3
    Good security research results in public findings. They do not seem to interested in creating public findings; so are they doing "good" security research? – Marcel Waldvogel Sep 15 '17 at 17:09
  • I blocked it by IP (35.225.221.87) and with `User-agent: ltx71 - (http://ltx71.com/) Disallow: /` in robots.txt) because it was crawling extremely long URLs on a WordPress site that nothing else seemed to be. – William Turrell May 29 '18 at 09:43
  • Nice reference to Sneakers 1992 – dvhh Aug 20 '19 at 13:15

1 Answers1

0

You can easily block this bot from .htaccess file on your web host.

 User-agent: ltx71 - (http://ltx71.com/)
 Disallow: *

Note: This bot is no use for your website.

Bbbb
  • 69
  • 2
  • 10