8

I have a virtual interface eth1:3 with IP address 222.192.124.3, I have set up an iptables rule to log packets coming on that interface using the interface's IP (I know iptables doesn't care about virtual interfaces labels), like this:

iptables -A INPUT -d 222.192.124.3 -j LOG --log-level warning --log-prefix "VIP3-IN: "

When I run tcpdump and send packets on this IP from another machine, I see them, so I assume they are received and handled properly by the kernel, but iptables never logs these packets, and when I run iptables -nvL, the packets count for that rule isn't incremented, like if they never hit the rule (or like if iptables doesn't even see packets coming on that interface).

I first thought of another rule matching the packet, and thus handling it before it hits the LOG rule, so I removed every iptables rule and added only the logging rule, with no more success.

Server is running on RHEL 6.2, with a 2.6.32 kernel, virtualized on VMware ESX.

Here is the full output of iptables -nvL:

Chain INPUT (policy ACCEPT 40 packets, 2891 bytes)
  pkts bytes target     prot opt in     out     source               destination
  0     0    LOG        all  --  *      *       0.0.0.0/0            222.192.124.3 LOG flags 0 level 4 prefix `VIP3-IN: '

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 34 packets, 3816 bytes)
  pkts bytes target     prot opt in     out     source               destination

And here is a sample of the tcpdump's output showing the incoming packets (tcpdump -n -nn -vvv -i eth1 "host 203.0.59.135"):

10:26:01.259409 IP (tos 0x50, ttl 121, id 26746, offset 0, flags [DF], proto TCP (6), length 52)
    203.0.59.135.62332 > 222.192.124.3.8888: Flags [S], cksum 0x8da8 (correct), seq 3373891789, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

And finally, the ifconfig eth1:3 output:

eth1:3    Link encap:Ethernet  HWaddr 00:50:56:AC:35:35
      inet addr:222.192.124.3  Bcast:222.192.127.255  Mask:255.255.252.0
      UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

UPDATE:

iptables diagram
(source: austintek.com)

Helped by the above diagram, I've setup iptables rules in different tables to see where the packets are going. I've come up with the following script:

IPT_FILTER="iptables -t filter"
IPT_MANGLE="iptables -t mangle"
IPT_NAT="iptables -t nat"

$IPT_FILTER -F
$IPT_FILTER -X
$IPT_FILTER -A INPUT -d 222.192.124.0/22 -j LOG --log-prefix "DEBUG filter: "
$IPT_FILTER -Z

$IPT_MANGLE -F
$IPT_MANGLE -X
$IPT_MANGLE -A PREROUTING -d 222.192.124.0/22 -j LOG --log-prefix "DEBUG mangle/prerouting: "
$IPT_MANGLE -A INPUT -d 222.192.124.0/22 -j LOG --log-prefix "DEBUG mangle/input: "
$IPT_MANGLE -Z

$IPT_NAT -F
$IPT_NAT -X
$IPT_NAT -A PREROUTING -d 222.192.124.0/22 -j LOG --log-prefix "DEBUG nat/prerouting: "
$IPT_NAT -Z

And it appears that packets go through the mangle/PREROUTING and nat/PREROUTING tables, but don't hit the mangle/INPUT table, so I guess it takes the "No" branch on the "Data for the firewall". And being by no means a network or system expert (at most a power-user), this is where I get lost and don't understand what's going on...

FINAL EDIT (solution)

As suggested by @nodens in their answer, the problem was caused by RPF being in "Strict" mode... So much headaches for such a simple setting...

mdeous
  • 398
  • 3
  • 11
  • I hate to dump yet more on you, but it's always better to **show** us what's going on, instead of **telling** us. Could you cut-and-paste in some of the `tcpdump` output that shows the packets that are not being detected? I **do** applaud your plan to strip out all the other firewall rules while chasing this issue down, though; that's good engineering thinking. – MadHatter Apr 04 '14 at 08:37
  • @MadHatter here you are – mdeous Apr 04 '14 at 08:46
  • Fair enough, and thank you. Could we also see the `tcpdump` command, as well as that line or two of output? This is a physical machine; there's no virtualisation anywhere in this question, yes? Can you reconfirm the zero packet counts on the rule (and also that the INPUT chain policy total **has** gone up)? As a final check, any chance of the `ifconfig` output showing the interface? – MadHatter Apr 04 '14 at 08:49
  • forgot to mention, server is virtualized on ESX. Added that info and the ifconfig output to the question. The total packets count for the INPUT table goes up, but as there are other interfaces, and traffic coming on them, I can't distinguish if the incrementation is caused by my packets or others – mdeous Apr 04 '14 at 08:52
  • Ah, then (for me, at least) all bets are off. There's so much weirdness that can happen at the virtual switching level, that it can baffle the mind - and cause things just like this. I would be glaring accusingly at the virtualisation level, with the assistance of my support contract, and not looking past that until it was squarely off the hook. – MadHatter Apr 04 '14 at 08:54
  • In the light of your update: let me set your mind at rest. These packets ought to hit the filter table's `INPUT` chain. On a physical box with a very similar setup, they most certainly **do** hit the `INPUT` chain. I don't think there's anything wrong with your understanding of `iptables`, which is why I'm **very** suspicious of the virtualisation. – MadHatter Apr 04 '14 at 11:22
  • Since you guess it takes the "No" branch, you should test that hypothesis by inserting a `LOG` rule in the `mangle` `FORWARD` chain. – kasperd Apr 04 '14 at 12:19
  • Hi there :) Can we have also an output of the following : "sysctl net.ipv4.ip_forward" and "sysctl -a|fgrep .rp_filter" "netstat -ltp". Thanks – Adrien M. Apr 05 '14 at 12:04
  • Thanks for your help guys, the problem was caused by RPF, as suggested by @nodens – mdeous Apr 10 '14 at 14:13

2 Answers2

3

Are you sure there is no matching rule in other tables, like mangle or nat ? You could also try to log every paquet coming in INPUT, or even better, try the TRACE target in raw table (PREROUTING chain), if that is available on RHEL 6

Edit (I can't add a comment yet) :

OK, so the packet hit the interface, but it's not considered as a one the should be treated locally. Check your routing table, maybe you don't have a scope link route to this network, or the packet is dropped by the kernel because of RPF filters (this could happen depending of the network topology) : check https://access.redhat.com/site/solutions/53031

nodens
  • 46
  • 3
  • yup, checked every other tables to make sure they are empty. I added "a few" more details to my question – mdeous Apr 04 '14 at 10:43
  • Oh, finally now I can add comments... OK, so the packet hit the interface, but it's not considered as a one the should be treated locally. Check your routing table, maybe you don't have a scope link route to this network, or the packet is dropped by the kernel because of RPF filters (this could happen depending of the network topology) : check https://access.redhat.com/site/solutions/53031 – nodens Apr 04 '14 at 19:41
  • arghh, didn't pay attention to the RPF part of your answer... THAT was the problem... Thank you! – mdeous Apr 10 '14 at 14:12
0

Virtual interface don't exist at the kernel level. They are aliases put on addresses by ifconfig. They should not be used since the end of the previous millennium.

Regarding iptables. As this is not an interface at kernel level you have to filter with the real interface name. In your case eth1. If you wish to be strict you will need to filter by source network if possible to mimic what you were trying to do by using the virtual interface name.

Eric Leblond
  • 11
  • I know that, that's why I used the interface's address instead – mdeous Apr 04 '14 at 09:21
  • Eric, what you say may be true from a "*best practice: the ip command must do everything*" standpoint, but as a statement of what actually works under CentOS 6, it's false. I've just tested on a C6.5 box with old-style virtual interfaces, and a tcpdump command constructed identically (save for address) to the OP's, and it **does** capture the traffic. I note also that you say the OP will have to filter with the real interface name, and that (s)he already is (with tcpdump) and specifies no interface at all (with iptables). – MadHatter Apr 04 '14 at 09:21