[asterisk-users] Slightly OT: trying to mangle packets from Asterisk for a multiple ISP setup (reward)

Mike list at virtutel.ca
Tue Jun 1 12:40:47 CDT 2010 

Hi Jimmy,

The rule is:

default via 22.22.22.22 dev eth1 src 22.22.22.21 

The src *should* change the originating IP address. It works for all other
statically defined routes with IP addresses.

Again:
32764: from all to 66.77.77.77 lookup test (that works fine)
32765: from all fwmark 4 lookup test  (this doesn't. It does seem to go in
that table, but the src isnt modified)

I just realized this is where I am stuck, the packets DO fall on the right
routing table when fwmark is used (they do go out of the right device, eth1,
which isnt the default one), but the only thing that is different is that
the src IP isnt modified as it is when the matching rule is based on IP.

IN summary, I can see the packets leaving via eth0 or eth1 depending on
whether I delete the fwmark 4 rule or add it back in.  So it`s definitely
eing matched and going into the right routing tables

Is there any reason who a fwmark matching rule acts differently when it
comes to rewriting the src address ? Google can`t help, I`m certainly not
the only one who's ever tried that.


Mike



> -----Original Message-----
> From: asterisk-users-bounces at lists.digium.com [mailto:asterisk-users-
> bounces at lists.digium.com] On Behalf Of Jimmy Godbout
> Sent: Tuesday, June 01, 2010 12:01
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [asterisk-users] Slightly OT: trying to mangle packets from
> Asterisk for a multiple ISP setup (reward)
> 
> Mike,
> 
> I don't see the outgoing rule in your description. How does your system
> knows how to change the outgoing address to match the originating nic ?
> 
> i.e.
> 
> 0: from all lookup local
> 32766: from all lookup main
> 32767: from all lookup default
> 
> ip rule add from 10.10.70.38/32 table admin
> ip rule add to 10.10.70.38/32 table admin
> 
> 0: from all lookup local
> 32764: from all to 10.10.70.38 lookup admin
> 32765: from 10.10.70.38 lookup admin  <=======================
> 32766: from all lookup main
> 32767: from all lookup default
> 
> Jimmy
> 
> > -----Original Message-----
> > From: list at virtutel.ca
> > Sent: Tue, 01 Jun 2010 10:58:52 -0400
> > To: asterisk-users at lists.digium.com
> > Subject: Re: [asterisk-users] Slightly OT: trying to mangle packets from
> > Asterisk for a multiple ISP setup (reward)
> >
> > Thanks Joe,
> >
> > They are on different segments.  Those two NICs share nothing but the
> > server.
> >
> > But more to the point, it doesn't explain why a simple routing rule
> > matching
> > the destination by IP address works wonderfully, but not one where I
> > match a
> > fwmark that has been set (apparently correctly according to my logging)
> > with
> > iptables.
> >
> > Mike
> >
> >> -----Original Message-----
> >> From: asterisk-users-bounces at lists.digium.com [mailto:asterisk-users-
> >> bounces at lists.digium.com] On Behalf Of Joe Freeman
> >> Sent: Tuesday, June 01, 2010 10:56
> >> To: asterisk-users at lists.digium.com
> >> Subject: Re: [asterisk-users] Slightly OT: trying to mangle packets
from
> >> Asterisk for a multiple ISP setup (reward)
> >>
> >> The simple fix your missing is to simply put your NICs on different
> >> layer 3 segments.
> >>
> >> In a configuration wherein multiple adapters are bound to the same
layer
> >> 3 network (subnet), most IP stacks will only send traffic out the first
> >> NIC to bind to that network. I've seen this many times in data centers
> >> where server guys configure the NICs in the same subnet for "load
> >> balancing".
> >>
> >> With some IP stacks, even arp responses are only sent from the first
NIC
> >> to bind to the subnet. This creates problems at layer 2, especially in
> >> large traffic volume situations. Since a technically correct arp
> >> response is sent - the mac address in the response is for the NIC to
> >> which the arp request was sent - from the first NIC to bind to the
> >> subnet, the switch(es) never actually learn where the MAC addresses for
> >> the other NICs are located. This creates a situation where all traffic
> >> for these NICs is broadcast to all ports in the VLAN.
> >>
> >> Joe
> >>
> >>
> >> On 6/1/2010 10:27 AM, Mike wrote:
> >>> Hi,
> >>>
> >>> Reward offered: 50$ (paypal), and I am sure this is a ridiculous thing
> >>> I
> >>> have missing.
> >>>
> >>> My goal: On a 2 NIC Asterisk box, to send packets that came in
Asterisk
> >>> on NIC1 back to NIC 1, and NIC2 back to NIC 2. (basically, send them
> >>> back the same way they came from).
> >>>
> >>> I have been doing what was recommended to me and mangling packets left
> >>> and right. I have reached a point where I am stuck, and can`t imagine
> >>> why this last little step isnt working.
> >>>
> >>> As you know, Asterisk sends all packets "from" the default IP (in my
> >>> case, NIC 1 IP). So connections to NIC 1 work fine, to NIC 2 they
> >>> don`t.
> >>> I therefore put in some routing rules to help me. Some example, a
phone
> >>> (remote PBX setup) coming in from 65.77.77.77 works fine because of
> >>> these (slightly obfuscated by changing IPs shown) routing rules:
> >>>
> >>> ip rule show:
> >>>
> >>> 0: from all lookup 255
> >>>
> >>> 32759: from all fwmark 0x64 lookup ISP2 (<----- this is key to my
> >>> issue)
> >>>
> >>> 32760: from all to 65.77.77.77 lookup ISP2
> >>>
> >>> 32766: from all lookup main
> >>>
> >>> 32767: from all lookup default
> >>>
> >>> ip route show table ISP2:
> >>>
> >>> default via 22.22.22.22 dev eth1 src 22.22.22.21
> >>>
> >>> BUT I can't reliably know where the phones come from (long story), or
> >>> what IP they use (ISP1 or ISP2) to connect to me. So instead, I have
> >>> done this with iptables:
> >>>
> >>> MARK all -- anywhere STRING match "22.22.22.21" ALGO name bm TO
> >>> 65535MARK set 0x64
> >>>
> >>> Basically saying to mark all packets that have the string
"22.22.22.21"
> >>> in it's SIP content (meaning they came in on NIC2 originally because
> >>> the
> >>> phone registered to 22.22.22.21) with mark 0x64. And that works fine,
> >>> because another rule that LOGs these marked packets is logging them
> >>> correctly.
> >>>
> >>> Because of my above routing rules, packets going out marked with 0x64
> >>> or
> >>> those going to 65.77.77.77 should go to the same ip route (route table
> >>> ISP2). Mysteriously, I see that packets going to 65.77.77.77 (using
> >>> wireshark) are correctly mangled as coming from 22.22.22.21, but not
> >>> those marked with 0x64. Those still go out through the default routing
> >>> table.
> >>>
> >>> What the heck am I missing? I believe I have done my homework, but
> >>> there
> >>> is no more door left to bang my head on.
> >>>
> >>> Mike
> >>>
> >>
> >>
> >> --
> >> _____________________________________________________________________
> >> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
> >> New to Asterisk? Join us for a live introductory webinar every Thurs:
> >>                http://www.asterisk.org/hello
> >>
> >> asterisk-users mailing list
> >> To UNSUBSCRIBE or update options visit:
> >>    http://lists.digium.com/mailman/listinfo/asterisk-users
> >
> >
> > --
> > _____________________________________________________________________
> > -- Bandwidth and Colocation Provided by http://www.api-digital.com --
> > New to Asterisk? Join us for a live introductory webinar every Thurs:
> >                http://www.asterisk.org/hello
> >
> > asterisk-users mailing list
> > To UNSUBSCRIBE or update options visit:
> >    http://lists.digium.com/mailman/listinfo/asterisk-users
> 
> ____________________________________________________________
> FREE 3D EARTH SCREENSAVER - Watch the Earth right on your desktop!
> Check it out at http://www.inbox.com/earth
> 
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
> New to Asterisk? Join us for a live introductory webinar every Thurs:
>                http://www.asterisk.org/hello
> 
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users

More information about the asterisk-users mailing list