0

I have read some answers here about blocking IP address ranges, and have already used iptables for this purpose before. It is suggested to use ipset in combination with iptables.

I have only installed ipset but have not configured it yet.

I found this site ip2location.com to generate a list of IPs to ban by country. I selected the 5 countries that target our sites regularly, but the list is huge, 256000 rows.

  • Would this massive list slow down my server when using ipset (before attempting this using only IPtables I questioned such a large file might slow performance.).
  • If that is the case, what is the way to do this? At the moment I use fail2ban but I do not think the configuration for nginx is correctly setup (I assume regex). In any case, I need a more robust way.
  • Finally, I do not claim to understand CIDR enough to make this list smaller (aggregate similar IP ranges if possible).

For instance, there are several /21 entries :

185.179.152.0/22

An online tool shows this resolves to : 185.179.152.0 to 185.179.155.255

I do not think there is any easy way to make the entries less, so any advice regarding implementation and performance issues please.

CvRChameleon
  • 103
  • 1
  • 3

3 Answers3

2

There is a command line utilty named aggregate. It takes a list of CIDR netblocks and aggregates consecutive blocks into the corresponding larger block. It also removes redundant netblocks.

For example:

$ aggregate -q << EOF
> 192.168.0.0/24
> 192.168.1.0/24
EOF
192.168.0.0/23

Feed it a text file containing only your CIDR blocks and it will attempt to aggregate them, reducing the size of the list.

From the man page:

DESCRIPTION
       Takes  a list of prefixes in conventional format on stdin, and performs
       two optimisations to attempt to reduce the length of the prefix list.

       The first optimisation is to remove any supplied prefixes which are su‐
       perfluous because they are already included in another supplied prefix.
       For example, 203.97.2.0/24 would be removed if 203.97.0.0/17  was  also
       supplied.

       The  second  optimisation identifies adjacent prefixes that can be com‐
       bined under a single, shorter-length prefix. For example, 203.97.2.0/24
       and 203.97.3.0/24 can be combined into the single prefix 203.97.2.0/23.

aggregate is packaged in most major Linux distributions, including Ubuntu.

(Note that I pulled a list from that web site and tried to aggregate them and nothing happened, so they may already be aggregated. You can certainly use more than one ipset, which is probably the best thing to do here.)

Michael Hampton
  • 244,070
  • 43
  • 506
  • 972
  • Ah this is definitely something I was looking for before. But yeah I think that website already aggregated things, as mentioned I tried a few online calculators and it gave me large ranges for one CIDR entry. Will run this definitely, and use multiple ipset lists. – CvRChameleon Jan 12 '21 at 08:20
1

Usually the max length of an ipset list is 65536 elements, so you might have to use a separate list for each country you want to block.

Using a hash:net set you can directly add the CIDR records you want to ban. You might want to check https://www.ipdeny.com/ipblocks/ for country level blocks

As for your questions

  • ipset shouldn't slow your system significantly - it will use some memory to keep the sets, but otherwise the load shouldn't be noticeable
  • fail2ban is good to keep, since the attackers can use cloud/vps servers in any country

Finally there are a lot of similar questions about using iptables with ipset to block specific countries, so I won't go into iptables setup details - just check https://askubuntu.com/questions/868334/block-china-with-iptables or similar

Dobromir Velev
  • 373
  • 2
  • 6
0

Could you use the xt_geoip xtables addon instead?

geoip vs ipset xt_geoip uses the (probably) most efficient format, a (non-compressed) packed blob. Loading one country into the kernel costs as much as the file on disk.

Since ipset does not support arbitrary IPaddr–IPaddr ranges, one would need to approximate that using, for example, multiple Network/Prefixlength entries. Furthermore, if a hash set type is used, you can assume that, by the nature of hashes and/or trees, some buckets remain empty and/or additional metadata is required. The memory footprint with an ipset-based geoip thus is naturally larger. User reports1 indicate it can become two orders of magnitude higher in certain cases (iptreemap).

xt_geoip's lookup time is O(log2(ranges)), so to lookup an address within 20,000 ranges, at most 15 iterations each with address comparisons (at most 3) are required. ipset uses Jenkins3 for hashing, which has a certain time cost of its own.

No empirical timing tests have been conducted so far.

Mark Wagner
  • 18,019
  • 2
  • 32
  • 47