[Asterisk-Dev] TCP/IP Gigabit Ethernet accel. question

[John Todd]

At 8:36 PM -0500 on 2/12/05, alex at pilosoft.com wrote:
>On Sat, 12 Feb 2005, John Todd wrote:
>
>>  In both of these circumstances, it seems that once some minor
>>  Asterisk issues were worked out that what was left to do was figure
>>  out how fast packets could be shuffled from one port to the other if
>>  RTP was being proxied as well as the SIP sessions.  Once that value
>>  was determined, the theoretical limit of number of possible channels
>>  (using, let's say, G.711) could be determined by figuring out the
>>  packet rate and interface limits, and the limit was not so much
>>  Asterisk's ability to move packets in an RTP-forwarding environment.
>I strongly doubt it. Asterisk is heavily involved with the packets,
>meaning that for each packet, there's userspace code that has to deal with
>it and find where does it go. (I.E. this isn't all in-kernel).
>
>>  Much of that burden might be TCP/IP stack overhead, and it seems there
>>  might be some ways to work around some of this overhead.  (I'm going to
>>  say at the moment that codec transcoding is out of the realm of
>>  discussion as far as this discourse is concerned, since that is a
>>  question of a different nature.)
>I can easily route, without trying very hard, 500kpps of normal traffic
>under linux. At 20ms RTP packets, it'd be around 25000 concurrent
>sessions. I doubt Asterisk can come close to that, and at best case it
>would be an order of magnitude off.
>
>>  This puzzlement over packet forwarding rate is not strictly an academic
>>  exercise to keep me busy on a rainy afternoon.  Asterisk has some
>>  excellent capabilities for performing RTP forwarding for a variety of
>>  reasons: topology disguise, SIP normalization (for some value of
>>  "normalize"), header re-writes, NAT traversal, or relay for QoS reasons
>>  (parallel network overlay and exit.)  Looking at port cost of other
>>  SBCs, it seems that Asterisk might be a reasonable choice for low-cost
>>  and high-flexibility, if the system can create fast enough call setup
>>  and also then handle RTP at a reasonable rate.
>Packet forwarding rate in kernel has really nothing to do with RTP
>forwarding that is done by Asterisk.
>
>>  blah blah blah - I lead with too much intro information.  Let's get to
>>  the "-dev" part of this, shall we?  A hardware manufacturer has
>>  mentioned to me that they have a TCP/IP stack and hardware interface
>>  card (gig e) which will improve performance up to 70% in a way that is
>>  transparent to the application.  It's a series of patches on RH9.0 which
>>  provides this capability.  The card has two Gig-E ports, and one would
>>  assume that an administrator would have two "sides" of their network
>>  (public/private? SIP/H323?) balanced, one to a port.
>I would suppose those cards are what's called "TCP Offload Engines" -
>meaning instead of kernel TCP code doing reassembly, card does it.
>
>Now, these will be somewhat useless to Asterisk because:
>a) RTP is UDP, not TCP, thus it won't benefit from it
>
>b) Even for TCP, there are major doubts where TOE helps at all. There are
>a number of issues with it, mostly, the money spent on TOE-capable card is
>much better spent by upgrading your general-purpose CPU. Also, kernel
>really doesn't like having its TCP functions being stolen. TCP reassembly
>is only a *tiny* part of a overall packet processing load, so saving 3% of
>your CPU utilization by spending 500$ on such a card is pointless.
>
>>  The hardware manufacturer has obliquely offered to allow a test of this
>>  card in a suitably high-volume environment.  At the moment, I don't have
>>  the time or the test lab at my disposal to try with a large number of
>>  channels (>1000) as an experiment.  Does anyone want to step forward to
>>  commit some time and energy to perhaps getting Asterisk more into the
>>  "carrier-class" sandbox? (<cough>OSDN<cough>)  If you are serious,
>>  please let me know and I'll introduce you to the person who has
>>  corresponded with me for this test inquiry.
>  >
>>  http://www.sbei.net/Products/LAN/lanPCI-2GC.htm
>That card isn't even a TOE engine. Just a regular sundance chip ethernet
>card. It might have some hardware support for .1q and .1ad, but that is
>not even useful in this application.
>
>-alex

My apologies.  I listed the wrong card.  Here is the card that was mentioned:

http://www.sbei.net/Products/SpecialtyCustom/toePCI-2Gx.htm

However, in light of the above discussions (and Kevin Fleming's 
similar comments) I'm less inclined to think that it's going to be a 
major benefit.  Most of the benefit of these types of cards seems to 
come from their TCP offloading, and RTP's UDP structure would only 
benefit from checksum generation, which seems to be a minor (if 
negligible?) benefit.

As to the comment about "25000" sessions with Asterisk: well, this 
puts us back to the benchmarking quandry that Asterisk has suffered 
with for so long... being as there is very little of it.  I'm not 
suggesting that the 25000 session number is wrong, just that I'd love 
to know what the actual number _is_ after testing.  Enterprise users 
don't typically like phrases like "Theoretically, it should be fine 
for your call volume, but no, I have no idea how many sessions it 
will handle."

I keep looking for "magic tricks" which will improve Asterisk's 
performance, for two reasons: 1) so they can be proven to work, or 2) 
so they can be dis-proven to work, but will force someone to publish 
the "breaking point" of the system under more sets of circumstances. 
In the case of #2, then it becomes a more simple issue to repair the 
bug/limit when one knows it exists.

JT