Adding sleep / delay between iptables blocks in shell script

Question

On one server, we're trying to block any access for the country China.

We wish to implement this with iptables rather than via .htaccess.

The problem we're experiencing though, is that when adding multiple iptables at once (and we're talking over 1000 ip subnets for China here), one will fail (with error 'iptables: Unknown error 18446744073709551615' and then subsequently all others after will fail) meaning we have to iptables -F in order to continue adding these rules.

Some testing concludes that the issue seems to be when trying to run lots at once (or even 10+ at a time seems to throw the error) rather than the server just not liking some specific rules.

So I wondered if trying to add a sleep (or some other delay) between each iptables rule running may improve the process?

We've attempted numerous ways of implementing these iptables nowbut we'd like, if possible, to use this script from nixCraft which can be seen below.

#!/bin/bash
### Block all traffic from AFGHANISTAN (af) and CHINA (CN). Use ISO code ###
ISO="af cn"

### Set PATH ###
IPT=/sbin/iptables
WGET=/usr/bin/wget
EGREP=/bin/egrep

### No editing below ###
SPAMLIST="countrydrop"
ZONEROOT="/root/iptables"
DLROOT="http://www.ipdeny.com/ipblocks/data/countries"

cleanOldRules(){
$IPT -F
$IPT -X
$IPT -t nat -F
$IPT -t nat -X
$IPT -t mangle -F
$IPT -t mangle -X
$IPT -P INPUT ACCEPT
$IPT -P OUTPUT ACCEPT
$IPT -P FORWARD ACCEPT
}

# create a dir
[ ! -d $ZONEROOT ] && /bin/mkdir -p $ZONEROOT

# clean old rules
cleanOldRules

# create a new iptables list
$IPT -N $SPAMLIST

for c  in $ISO
do
    # local zone file
    tDB=$ZONEROOT/$c.zone

    # get fresh zone file
    $WGET -O $tDB $DLROOT/$c.zone

    # country specific log message
    SPAMDROPMSG="$c Country Drop"

    # get 
    BADIPS=$(egrep -v "^#|^$" $tDB)
    for ipblock in $BADIPS
    do
       $IPT -A $SPAMLIST -s $ipblock -j LOG --log-prefix "$SPAMDROPMSG"
       $IPT -A $SPAMLIST -s $ipblock -j DROP
    done
done

# Drop everything 
$IPT -I INPUT -j $SPAMLIST
$IPT -I OUTPUT -j $SPAMLIST
$IPT -I FORWARD -j $SPAMLIST

# call your other iptable script
# /path/to/other/iptables.sh

exit 0

I'm not particularly familiar with shell scripts however and am unsure at what point in this script I could add some sort of delay between iptables processing, if even possible at all. Please could someone point me in the right direction? Thanks.

dsmsk80 · Accepted Answer · 2013-08-27T13:24:05.677

Netfilter/iptables doesn't scale well when used with higher number of rules as they are supposed to be matched sequentially. But since the Linux kernel 2.6.36, there is a new feature called IP sets which helps to eliminate such rules by using hashing techniques. Briefly, how it works:

1) create an IP set

ipset create set01 hash:net

2) add IP ranges to the set

ipset add set01 192.168.1.0/24
ipset add set01 192.168.2.0/24
ipset add set01 192.168.3.0/24
ipset add set01 192.168.4.0/24
...
...

3) connect the set with netfilter/iptables rule which rejects any packet with source IP address from the set set01

iptables -A INPUT -m set --match-set set01 src -j REJECT

As you can see, instead of creating 4 almost the same iptables rules I created one IP set set01, then I added some IP ranges to it (just an example, in reality, there may be millions of them) and finally I blocked them by single iptables rule. So instead of maintaining a long list of similar iptables rules, create an IP set(s) according the zones files and then block it in one step.

Please check the related ipset man page for more details.

score 1 · Answer 2 · answered Aug 27 '13 at 13:45

I agree with dsumsky that you might need a new approach. But to delay 1 second for every 4 entries loaded, the code to insert would be something like this:

i=0  # delay iterator

BADIPS=$(egrep -v "^#|^$" $tDB)
for ipblock in $BADIPS
do
   # delay to help rules load properly.
   if [[ $i == 5 ]]
   then
         sleep 1
         i=0
   fi

   i=$((i+1))

   $IPT -A $SPAMLIST -s $ipblock -j LOG --log-prefix "$SPAMDROPMSG"
   $IPT -A $SPAMLIST -s $ipblock -j DROP
done

You can experiment with the code with a block of code like this:

 #!/bin/bash
 i=0
 while [ 1 ]
 do
    i=$((i+1))
    if [[ $i == 5 ]]
    then
            sleep 1
            i=0
    fi


    echo hello $i
done

Adding sleep / delay between iptables blocks in shell script

2 Answers2