[asterisk-users] Slightly OT: trying to mangle packets from Asterisk for a multiple ISP setup (reward)

[Joe Freeman]

The simple fix your missing is to simply put your NICs on different 
layer 3 segments.

In a configuration wherein multiple adapters are bound to the same layer 
3 network (subnet), most IP stacks will only send traffic out the first 
NIC to bind to that network. I've seen this many times in data centers 
where server guys configure the NICs in the same subnet for "load 
balancing".

With some IP stacks, even arp responses are only sent from the first NIC 
to bind to the subnet. This creates problems at layer 2, especially in 
large traffic volume situations. Since a technically correct arp 
response is sent - the mac address in the response is for the NIC to 
which the arp request was sent - from the first NIC to bind to the 
subnet, the switch(es) never actually learn where the MAC addresses for 
the other NICs are located. This creates a situation where all traffic 
for these NICs is broadcast to all ports in the VLAN.

Joe


On 6/1/2010 10:27 AM, Mike wrote:
> Hi,
>
> Reward offered: 50$ (paypal), and I am sure this is a ridiculous thing I
> have missing.
>
> My goal: On a 2 NIC Asterisk box, to send packets that came in Asterisk
> on NIC1 back to NIC 1, and NIC2 back to NIC 2. (basically, send them
> back the same way they came from).
>
> I have been doing what was recommended to me and mangling packets left
> and right. I have reached a point where I am stuck, and can`t imagine
> why this last little step isnt working.
>
> As you know, Asterisk sends all packets "from" the default IP (in my
> case, NIC 1 IP). So connections to NIC 1 work fine, to NIC 2 they don`t.
> I therefore put in some routing rules to help me. Some example, a phone
> (remote PBX setup) coming in from 65.77.77.77 works fine because of
> these (slightly obfuscated by changing IPs shown) routing rules:
>
> ip rule show:
>
> 0: from all lookup 255
>
> 32759: from all fwmark 0x64 lookup ISP2 (<----- this is key to my issue)
>
> 32760: from all to 65.77.77.77 lookup ISP2
>
> 32766: from all lookup main
>
> 32767: from all lookup default
>
> ip route show table ISP2:
>
> default via 22.22.22.22 dev eth1 src 22.22.22.21
>
> BUT I can't reliably know where the phones come from (long story), or
> what IP they use (ISP1 or ISP2) to connect to me. So instead, I have
> done this with iptables:
>
> MARK all -- anywhere STRING match "22.22.22.21" ALGO name bm TO
> 65535MARK set 0x64
>
> Basically saying to mark all packets that have the string "22.22.22.21"
> in it's SIP content (meaning they came in on NIC2 originally because the
> phone registered to 22.22.22.21) with mark 0x64. And that works fine,
> because another rule that LOGs these marked packets is logging them
> correctly.
>
> Because of my above routing rules, packets going out marked with 0x64 or
> those going to 65.77.77.77 should go to the same ip route (route table
> ISP2). Mysteriously, I see that packets going to 65.77.77.77 (using
> wireshark) are correctly mangled as coming from 22.22.22.21, but not
> those marked with 0x64. Those still go out through the default routing
> table.
>
> What the heck am I missing? I believe I have done my homework, but there
> is no more door left to bang my head on.
>
> Mike
>