[asterisk-users] Scaling Asterisk: Dual-Core CPUs not yielding gains at high call volumes

[Matthew J. Roth]

List users,

Using Asterisk in an inbound call center environment has led us to 
pushing the limits of vertical scaling.  In order to treat each caller 
fairly and to utilize our agents as efficiently as possible, it is 
desirable to configure each client as a single queue.  As far as I know, 
Asterisk's queues cannot be distributed across servers, so the size of 
the largest queue we service is our vertical scaling goal.  In our case, 
that queue must be able to hold in excess of 300 calls regardless of 
their makeup (ie. number of calls in queue vs. number of calls connected 
to an agent).  In reality, we are servicing more than one client on our 
server, so on busy days the total number of calls we're handling is 
greater than 300.

Recently, we were pushing our server to almost full CPU utilization.  
Since we've observed that Asterisk is CPU bound, we upgraded our server 
from a PowerEdge 6850 with four single-core Intel Xeon CPUs running at 
3.16GHz, to a PowerEdge 6850 with 4 dual-core Intel Xeon CPUs running at 
3.00GHz.  The software installed is identical and a kernel build 
benchmark yielded promising results.  The new dual-core server ran 
roughly 80% faster, which is about what we expected.

As far as Asterisk is concerned, at low call volumes the dual-core 
server outperforms the single-core server at a similar rate.  I'm 
working on a follow-up post that will demonstrate this with some 
benchmarks for a small number of calls in various scenarios on each 
machine.  However, to our surprise as the number of concurrent calls 
increases, the performance gains begin to flatten out.  In fact, it 
seems that somewhere between 200 and 300 calls, the two servers start to 
exhibit similar idle times despite one of them having twice as many cores.

Once I collect the data, I will add a second follow-up post with a 
performance curve tracking the full range of call volumes we 
experience.  Unfortunately, from day to day there are some variables 
that I'm sure affect performance, such as the number of agents logged in 
and the makeup of the calls.  I'll do my best to choose a sample size 
that smooths out these bumps.

In the meantime, I'm looking for insights as to what would cause 
Asterisk (or any other process) to idle at the same value, despite 
having similar workloads and twice as many CPUs available to it.  I'll 
be working on benchmarking Asterisk from very low to very high call 
volumes so any suggestions or tips, such as how to generate a large 
number of calls or what statistics I should gather, would also be 
appreciated.

Thank you,

Matthew Roth
InterMedia Marketing Solutions
Software Engineer and Systems Developer