[Asterisk-Dev] TCP/IP Gigabit Ethernet accel. question

[alex at pilosoft.com]

On Sat, 12 Feb 2005, John Todd wrote:

> In both of these circumstances, it seems that once some minor 
> Asterisk issues were worked out that what was left to do was figure 
> out how fast packets could be shuffled from one port to the other if 
> RTP was being proxied as well as the SIP sessions.  Once that value 
> was determined, the theoretical limit of number of possible channels 
> (using, let's say, G.711) could be determined by figuring out the 
> packet rate and interface limits, and the limit was not so much 
> Asterisk's ability to move packets in an RTP-forwarding environment. 
I strongly doubt it. Asterisk is heavily involved with the packets, 
meaning that for each packet, there's userspace code that has to deal with 
it and find where does it go. (I.E. this isn't all in-kernel).

> Much of that burden might be TCP/IP stack overhead, and it seems there
> might be some ways to work around some of this overhead.  (I'm going to
> say at the moment that codec transcoding is out of the realm of
> discussion as far as this discourse is concerned, since that is a
> question of a different nature.)
I can easily route, without trying very hard, 500kpps of normal traffic
under linux. At 20ms RTP packets, it'd be around 25000 concurrent
sessions. I doubt Asterisk can come close to that, and at best case it 
would be an order of magnitude off.

> This puzzlement over packet forwarding rate is not strictly an academic
> exercise to keep me busy on a rainy afternoon.  Asterisk has some
> excellent capabilities for performing RTP forwarding for a variety of
> reasons: topology disguise, SIP normalization (for some value of
> "normalize"), header re-writes, NAT traversal, or relay for QoS reasons
> (parallel network overlay and exit.)  Looking at port cost of other
> SBCs, it seems that Asterisk might be a reasonable choice for low-cost
> and high-flexibility, if the system can create fast enough call setup
> and also then handle RTP at a reasonable rate.
Packet forwarding rate in kernel has really nothing to do with RTP 
forwarding that is done by Asterisk. 

> blah blah blah - I lead with too much intro information.  Let's get to
> the "-dev" part of this, shall we?  A hardware manufacturer has
> mentioned to me that they have a TCP/IP stack and hardware interface
> card (gig e) which will improve performance up to 70% in a way that is
> transparent to the application.  It's a series of patches on RH9.0 which
> provides this capability.  The card has two Gig-E ports, and one would
> assume that an administrator would have two "sides" of their network
> (public/private? SIP/H323?) balanced, one to a port.
I would suppose those cards are what's called "TCP Offload Engines" - 
meaning instead of kernel TCP code doing reassembly, card does it. 

Now, these will be somewhat useless to Asterisk because: 
a) RTP is UDP, not TCP, thus it won't benefit from it

b) Even for TCP, there are major doubts where TOE helps at all. There are
a number of issues with it, mostly, the money spent on TOE-capable card is
much better spent by upgrading your general-purpose CPU. Also, kernel
really doesn't like having its TCP functions being stolen. TCP reassembly
is only a *tiny* part of a overall packet processing load, so saving 3% of
your CPU utilization by spending 500$ on such a card is pointless.

> The hardware manufacturer has obliquely offered to allow a test of this
> card in a suitably high-volume environment.  At the moment, I don't have
> the time or the test lab at my disposal to try with a large number of
> channels (>1000) as an experiment.  Does anyone want to step forward to
> commit some time and energy to perhaps getting Asterisk more into the
> "carrier-class" sandbox? (<cough>OSDN<cough>)  If you are serious,
> please let me know and I'll introduce you to the person who has
> corresponded with me for this test inquiry.
> 
> http://www.sbei.net/Products/LAN/lanPCI-2GC.htm
That card isn't even a TOE engine. Just a regular sundance chip ethernet
card. It might have some hardware support for .1q and .1ad, but that is 
not even useful in this application.

-alex