[Asterisk-Users] Asterisk Crashing (high load issues)

[steve at daviesfam.org]

On Wed, 9 Nov 2005, Kyle Hagan wrote:

>  We purchased a new Dual Xeon 3ghz, 2gb ram to upgrade our 3ghz Pentium 
> 1gb ram, that has been having load issues due to our growing company.
> 
> We are having problems... We use a predictive dialer that we custom 
> programmed in perl. It basically drops, moves, files into the callout 
> directory and uses queues to transfer to agents when someone picks up.
> 
> It has been working pretty good, except we now have 50+ dialers on the 
> system taking calls. The system dials 2-4 per available agent every 3-5 
> seconds based on, calls ringing and available agents. We can keep them 
> to about 8-20 seconds between calls. But the number of ringing lines is 
> causing load issues. Hence the new server.
> 
> We put Fedora Core 4 on with now problem. We were running 2 t1's in the 
> beginning of the day just to make sure the system was running good. We 
> finally put it on 8 t1's and the system ran great for about 4 hours. 
> Then the load started going up and up until the server just locked 
> completely. I could not get much information from the server. The lead 
> went to 170+ before it locked. Asterisk was showing 99% cpu usage at crash.
> 
>  I have some information that the log had in it just before the crash. 
> There was something about cpu3 soft lockup and page fault messages. If 
> someone can help I will post the log tomorrow when I get into work.
> We had to switch back to the old server with the load issues.
> 
>  Some other information about the servers follows:
> 
>  We are running a separate slim server to stream moh.
>  The predictive server is a separate pc connecting via manager interface 
> for agent information, available, busy and callerid of the person they 
> are talking to
>  We have a script (perl) running on the Asterisk server to move the 
> callout files into the callout directory that are created via a web POST 
> via apache, the script checks for files in a temp directory and move the 
> files into the callout directory.


Hi Kyle,

I'd simply say that you have overloaded that machine.

We use boxes like that for a similar outbound dial setup.  I don't think 
I'd attempt to go past 4 E1s (120 lines) which would be 5 T1s.

If the box is running hard like that the load average will sit around 7 or 
so, still fair amount of spare CPU but there is no way an Asterisk box 
will run well with the CPU anything like maxed out.

Our site has 250 agents or so and the work is currently spread over 6 
servers with 3 E1 PRIs on each.  Each box makes around 3000 to 5000 call 
attempts per hour.

If you are getting very high load average - are you recording calls?  It 
would REALLY not be a good idea to use the "m" option to Monitor to mix 
calls on the fly - the soxmix processes will accumulate and accumulate.

Your Perl dialler also needs to be more sympathetic to the machine 
capacity and back off when the server is getting overloaded.  Otherwise 
you are certain to drive it into the ground.

Steve