[asterisk-dev] Asterisk Load Performance

[Richard Mudgett]

On Tue, Jul 5, 2016 at 4:03 PM, Jonathan Rose <
jonathan.rose at motorolasolutions.com> wrote:

>
> On Tue, Jul 5, 2016 at 3:43 PM, Michael Petruzzello <
> michael.petruzzello at civi.com> wrote:
>
>> On Wed, Jun 29 at 11:14:04 AM, Richard Mudgett<rmudgett at digium.com
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__digium.com&d=CwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=50uagQBTpQAKCx3KjAwJcMd6ygCPToAyDAxH5npANtf7nLmyZ65ofHGUgyJr9BW8&m=RzNw4lQyfY2yXT49Ylv_v1goTGLiuwUxFtihFJb5GAs&s=P78xn7-C1l6ETO4vrsRQlsDcWLZFaEJEG71kAzrXHOU&e=>>
>> wrote:
>> > Each softmix bridge has only one thread performing all of the media
>> mixing
>> > for the bridge.  To
>> > get better mixing performance for such a large conference, you will
>> need to
>> > create several
>> > softmix bridges in a hierarchy with the bridges linked by local
>> channels.
>>
>> A bridge is only able to handle around 2000-2500 channels, so I created
>> 15 bridges with 14 channels bridging the bridges together.
>>
>> When doing this an error I see a lot is WARNING[98920]: channel.c:1101
>> __ast_queue_frame: Exceptionally long voice queue length queuing to
>> Local/**********@default-00000000;2, which then turns into WARNING[47525]:
>> pjproject:0 <?>:      sip_transactio .Unable to register INVITE transaction
>> (key exists) and ERROR[47525]: res_pjsip.c:2777 ast_sip_create_dialog_uas:
>> Could not create dialog with endpoint sippeer. Object already exists
>> (PJ_EEXISTS). Finally the following repeats over and over again, [Jun 30
>> 12:22:21] ERROR[84189][C-00000958]: netsock2.c:305 ast_sockaddr_resolve:
>> getaddrinfo("domain.name
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__domain.name&d=CwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=50uagQBTpQAKCx3KjAwJcMd6ygCPToAyDAxH5npANtf7nLmyZ65ofHGUgyJr9BW8&m=RzNw4lQyfY2yXT49Ylv_v1goTGLiuwUxFtihFJb5GAs&s=0TFQxSqdacQKRz5CbZtpR8WMsE1K1ZdgwZDCoWQPAnU&e=>",
>> "(null)", ...): Temporary failure in name resolution
>> [Jun 30 12:22:21] WARNING[84189][C-00000958]: acl.c:800 resolve_first:
>> Unable to lookup 'domain.name
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__domain.name&d=CwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=50uagQBTpQAKCx3KjAwJcMd6ygCPToAyDAxH5npANtf7nLmyZ65ofHGUgyJr9BW8&m=RzNw4lQyfY2yXT49Ylv_v1goTGLiuwUxFtihFJb5GAs&s=0TFQxSqdacQKRz5CbZtpR8WMsE1K1ZdgwZDCoWQPAnU&e=>
>> '.
>>
>
The exceptionally long voice queue length messages can be a symptom of
thread
starvation as the Local channels frame queue has developed an excessive
backlog.

The forthcoming v13.10.0 release should indirectly take care of the EEXISTS
messages
as part of the https://issues.asterisk.org/jira/browse/ASTERISK-26088 fix.
Working on
that issue I saw the EEXISTS messages for REGISTER and SUBSCRIBE message
processing.  The issue was a result of the original message and
retransmissions getting
backlogged in the serializer/taskprocessor and responses sent using another
serializer.

Looks like your system's DNS resolver has gotten overwhelmed.


>
>> The last error just keeps on repeating and calls can no longer join (only
>> around 3,500 make it on before this starts to occur). Calling in manually I
>> receive an "all circuits are busy" message.
>>
>> I'm going to try halving the number of bridges, but is there anything
>> else I can do to improve performance? This seems to be the last hurdle to
>> use one server for 10,000 callers.
>>
>
> If you don't need all of your participants actually to be speaking at a
> time (and I hope not with that kind of volume), you could use holding
> bridges for the vast majority of the partipants. Link the bridges using a
> local channel with the Hold bridge side being set to use the 'announcer'
> bridge role and the hold bridge will effectively just be voiceless
> conference participants. If you want, you can listen for DTMF events to
> move the participants back and forth between the different bridges.
>

Wow.  Thanks Jonathan.  I hadn't thought of doing it that way.  That should
really drop the mixing load.
Probably should allow only ulaw or alaw (pick one) for all participants to
minimize translation costs.
One additional thing I should add is that those linking Local channel
bridges should just allow the
chosen alaw/ulaw to reduce translation to each participant in the holding
bridge.  The forthcoming
v13.10.0 adds the ability to specify formats when ARI originates a channel
(Local in this case) and
an originator channel is not available.  (See CHANGES file)

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20160705/25993b7c/attachment.html>