[asterisk-dev] Asterisk scalability

[Benny Amorsen]

John Todd <jtodd at digium.com> writes:

> In fact, it seems like a great idea in general to have multiple
> cards even under moderate load, since even with IRQ balancing (which
> apparently doesn't actually work) there is one CPU that takes all
> the interrupts from a single card. Plus, 802.3ad seems like a quick
> way to get some wiring redundancy even though you're keeping your
> other single points of failure intact (primary system, upstream
> switch.)

Your only challenge with bonding is to make sure that all streams are
kept on the same physical interface. I think that is done by default
in 802.3ad (based on a hash of the ip-adresses and ports involved), so
that ought to be easy. The alternative is multiqueue. In a year or two
all nics will be multiqueue capable.

I did a strace on Asterisk. It looks like it works this way:
A thread listens for a specific stream with recvfrom() and waits with
poll(). When a packet comes in, it does sendto(). That is 3 kernel
calls per packet. Ouch. It also costs two copies of the packet, but
copies are dirt cheap.

It's difficult to see how to do much better. splice() would be nice,
but doesn't work for UDP receive AFAIK. Multiple packets at once would
be another alternative, but I don't see any way to do that. It would
be tempting to add a kernel helper for the cases where the RTP is
bridged without changes apart from IP addresses and port numbers, but
I suppose that would be almost equivalent to implementing splice() for
UDP receive.

In general, all the performance work is done on TCP, and that doesn't
help VoIP. Right now the only way to handle a lot of RTP streams is to
reinvite them and let the kernel route them.


/Benny