Blocked from accessing website with EC2 Instance

Question

When I run this command on my Ubuntu 16.04 EC2 Instance:

curl -vvv https://www.oddsportal.com/site-map-active/

it returns

* connect to 188.92.41.44 port 443 failed: Connection timed out
* Failed to connect to www.oddsportal.com port 443: Connection timed out
* Closing connection 0
curl: (7) Failed to connect to www.oddsportal.com port 443: Connection timed out

However, on my local machine, it returns the expected HTML.

Is it true that AWS IP's are blacklisted from certain domains?

Yes, I have two domains, and I've blacklisted all IP blocks for EC2 due to excessive abuse, which I've reported to abuse@aws who did nothing about it. Actually, I'm a little surprised anyone is running a browser on those instances, I thought those were for servers. Certainly, regular user browsing did not take a hit after I blacklisted EC2. — Uri Raz, Dec 12 '22 at 15:46

score 1 · Answer 1 · answered Sep 10 '19 at 06:42

1

Is it true that AWS IP's are blacklisted from certain domains?

Yes, at least shouldn't surprise you.

One of several old jobs I've been involved with was involved with web scraping, and you get to know several degrees of scraping tolerance when dealing with the different cloud servers providers.

There (I mean at this shop), AWS was not only famous from old blacklisting experiences, but also as one of cloud servers providers which would be most wary of the facts hinting that their servers might being used for web scraping.

I would think of this thing as that from each vendor's control panel it sort of shines as a pretty obvious thing, and so somehow becomes a matter of company strategy.

All of this notwithstanding the fact any domain owner has the right to ban whatever address ranges they want.

You might want to game in other cloud providers to your mix and maybe you get a different blend of things.

answered Sep 10 '19 at 06:42

178024

36
3
15

Could you elaborate on a different blend of things? – frantic oreo Sep 10 '19 at 07:37
Imagine you wanted to download the whole Internet. As some providers' servers would be blocked from some sites, but no site ever gets to block every provider, in the end you have a big queue at home, and queue processing in each cloud provider. Even if most sites would block most providers, it is likely that the queue is going down, slower or faster. – 178024 Sep 10 '19 at 07:54
So there's the big queue, work distribution to queue processors, and a blend result from that work, which depends on which providers you use, which providers remotes blacklist, and which providers allow you emit a substantial amount of HTTP traffic. – 178024 Sep 10 '19 at 07:54

score 0 · Answer 2 · answered Sep 10 '19 at 06:42

0

I think the IP assigned to you might have been involved in some spam. So, that website might be blocking your IP. Contact AWS customer care for further information. Also, search for your IP on ABUSEDB. See if there is any abuse report against your IP

answered Sep 10 '19 at 06:42

Rahul Biswas

139
1
1
14

1

Spam sender black lists are typically only used to reject mail (SMTP) traffic , not other traffic. – HBruijn Sep 10 '19 at 06:57
Thanks @HBruijn I learnt something new today! – Rahul Biswas Sep 10 '19 at 14:34

score 0 · Answer 3 · answered Sep 10 '19 at 06:47

Is it true that AWS IP's are blacklisted from certain domains?

Yes.

Sites that offer region locked content will use a number of techniques to ensure such policies and a common one is use the source (IPv4) IP-address as a discriminator.

To such sites AWS IP-addresses typically represent users trying to hide their real IP address and/or also those IP’s are never the actual IP address of a human content consumer and therefore blocked.

———

That might not be the root of your problem though. Your instance may be configured with a security policy restring egress traffic for instance...

Blocked from accessing website with EC2 Instance

3 Answers3