[asterisk-dev] Asterisk Load Performance

[Michael Petruzzello]

It is very interesting how threading issues on both a stasis application
and Asterisk escalate each other. Using 15 websockets in one stasis
application and removing all thread locking from the application have made
the ARI messages flow smoothly. Right now I am using about 900 threads to
process messages from Asterisk and Asterisk has at least 320 in stasis,
though that can increase to infinity.

I have also disabled the channel_varset from stasis because it becomes
really unwieldy. When having thousands of callers in a bridge, every time a
channel is added to a bridge or removed, every channel receives a channel
var set message because of the BridgePeer variable.

As of now, I have two remaining problems:

1. At around having 5,000 channels in a bridge (whether majority are muted
or not), the audio breaks down. Anyone talking can only be heard in 3
second bursts approximately every 5-10 seconds. At 10,000 channels only
static can be heard in these 3 second bursts.

Is there anything I can optimize so that Asterisk can handle all these
channels in a bridge?

2. Every time a channel joins the bridge, the websocket responsible for
that channel is then subscribed to the bridge. Then any events that occur
on that bridge (such as another channel entering or exiting it) are sent to
that websocket. Because every websocket then ends up receiving these
messages, it defeats the point of having multiple websockets. To get around
this I have been unsubscribing the websockets from the bridge anytime a
channel from that websocket enters the bridge, but this isn't perfect as
timing is an issue.

Is there anyway to disable this automatic subscription behavior to a bridge?


*Michael J. Petruzzello*
Software Engineer
P.O. Box 4689
Greenwich, CT 06831
203-618-1811 ext.289 (office)
www.civi.com

On Tue, Jun 21, 2016 at 3:29 PM, Michael Petruzzello <
michael.petruzzello at civi.com> wrote:

> On Tue, Jun 21, 2016 at 12:16 PM, Richard Mudgett <rmudgett at digium.com>
> wrote:
> > The subm:devService-test-00000038 taskprocessor is servicing the stasis
> > message bus
> > communication with your devService-test ARI application.  Since each
> > taskprocessor is
> > executed by one thread, that is going to be a bottleneck.  One thing you
> > can try is to
> > register multiple copies of your ARI application and randomly spread the
> > calls to the
> > different copies of the application.  (devService-test1,
> > devService-test2,...)
>
> Ah, that explains it! Everything else has been running well in Asterisk as
> far as handling the actual channels and the SIP messaging with the large
> calls / second.
>
> I was thinking about the potential of parallelizing the stasis message bus
> communication to use multiple task processors, but that would introduce
> other issues. Messages would be sent out of order, and the ARI application
> would need to handle that.
>
> Your suggestion sounds like the best approach. That way I still have only
> one application with multiple connections to Asterisk. No need to have
> multiple applications and servers that would need to communicate together.
>
> Thank you for the insight.
>
> On Tue, Jun 21, 2016 at 1:03 PM, Matthew Jordan<mjordan at digium.com>
> wrote:
> > To follow up with Richard's suggestion:
> >
> > Events being written out (either over a WebSocket in ARI or over a
> > direct TCP socket in AMI) have to be fully written before the next
> > event is written. That means that the client application processing
> > the events can directly slow down the rate at which events are sent if
> > the process that is reading the event does not keep reading from the
> > socket as quickly as possible. You may already be doing this - in
> > which case, disregard the suggestion - but you may want to have one
> > thread/process read from the ARI WebSocket, and farm out the
> > processing of the events to some other thread/process.
>
> I am already doing this, but thank you for the suggestion. Database access
> really slows down the processing of events so I have had to do this from
> the start of my project.
>
>
> *Michael J. Petruzzello*
> Software Engineer
> P.O. Box 4689
> Greenwich, CT 06831
> 203-618-1811 ext.289 (office)
> www.civi.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20160629/49863307/attachment.html>