Linux router: ping doesn't route back

Solution 1:

From your question in the comments:

On the remote server I see requests and replies. But on the Debian router I don't see anything... on none of the interfaces! My guess is that now, the Ubuntu box is talking directly to the router on 192.168.1.1 THOUGH sending requests with IP 10.1.1.12, so it can't route back. But why??

From the Ubuntu server:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0 <---
0.0.0.0         10.1.1.1        0.0.0.0         UG    100    0        0 eth1

At the time you captured this routing table, you have a lower metric default through eth0 pointing to your router at 192.168.1.1 (i.e. not the debian machine). A lower metric default is always followed first, which means Ubuntu wants to send all non-connected traffic directly to 192.168.1.1.

When you have downtime available, please remove that default with

route del default gw 192.168.1.1 dev eth0

I'm still simmering on the bigger problem (original sniffer traces show ping replies on Ubuntu:eth1, but no pings accepted by the OS). Could you please ping from Ubuntu:eth1 and simultaneously capture on Debian:eth2 to demonstrate what is happening with NAT after you force the Ubuntu to send all traffic through the Debian again?

Solution 2:

Did you check if reverse path filtering is enabled on the Ubuntu box?

It's a sysctl setting (net.ipv4.conf.all.rp_filter), it will filter incoming packets if the source address is coming in on the "wrong" interface ( i.e. not the interface that the kernel would route it to )

You could also try net.ipv4.conf.all.log_martians=1 to try to see what's happening.

---

Solution 3:

The key to make this work is to create separate routing tables for the different interfaces, and tell the networking stack to use these routing tables instead of the default one.

In your case this should make ping -I eth2 8.8.8.8 work:

# register the 'foo' table name and give it id 1
echo '1 foo' >> /etc/iproute2/rt_tables

# setup routing table 'foo'
ip route add 192.168.1.0/24 dev eth2 src 192.168.1.10 table foo
ip route add default via 192.168.1.1 table foo

# use routing table 'foo' for address 192.168.1.10
ip rule add from 192.168.1.10 table foo

More information on routing for multiple uplinks can be found in the LARTC HOWTO: http://lartc.org/howto/lartc.rpdb.multiple-links.html