[asterisk-dev] Asterisk scalability

[Gregory Boehnlein]

> 'bonding' as it's called in linux does work. I did not test if we can
> handle the double ammount of calls, because the setups I did only
> handle like 100 to 150 concurrent calls. But I can see on the interface
level
> the both nics in the bonding device have roughly handled the same
> ammount of rx and tx packets.
> The two interfaces are connected to a cisco 3500 switch, and I did not
> configure the switch so I have no idea what knobs you have to turn
> there.

Not that this is Asterisk, but I have successfully used 802.3ad LAG with the
iSCSI Enterprise Target to build multi-gigabit / second iSCSI SANS and have
been able to saturate both Gig-E links easily. This is a very common
scenario when one is building a virtualization platform, and has it's own
set of tuning parameters.

However, the iSCSI Enterprise Target mailing list has a lot of high-level
developers that offer tips for tuning both the Linux kernel, as well as the
Ethernet drivers for best performance. As a result of following their
advice, and sticking w/ known good hardware (Intel Server Nics!) I've been
able to saturate Gig-E links w/ iSCSI Traffic between a handful of Vmware
ESX Servers and the SAN device. 

One of the particularly interesting things that I found through the process
of tuning a SAN implementation was that three main things impact performance
in additive ways.

1. Disabling Flow Control on the switch and Ethernet NICs which results in a
minor loss of top-end burst speed, but greatly reduces latency on packets
moving through the switch. The tradeoff is a higher load on the CPU and
Ethernet driver as it interrupts more frequently for I/O.

2. Changing the Linux kernel scheduler to make it a more responsive to I/O
requests and service things in a lower latency fashion.

3. Disabling and tuning NIC parameters such as Interrupt Coalescence.

One of the other things that comes into play is the actual load-balancing
implementation that the switch uses. On the Netgears that we use, it doesn't
start using the second pipe until the first one is saturated.