[asterisk-app-dev] High CPU in optimizable bridge cases

[Matthew Jordan]

On Mon, Dec 23, 2013 at 4:12 PM, Ben Klang <bklang at mojolingo.com> wrote:

> Hello all,
>
> I’m not sure if this is the right venue for this question, so please let
> me know if there is a better place to ask.
>
> We have been experimenting with using ConfBridge to reduce connect time
> for agents on an outbound dialer (reminder calls, in case you’re
> wondering). In a simulation, we parked 80 idle agents into 80 idle
> ConfBridge channels.  This is to say, 80 channels were sitting alone in
> separate rooms with no mixing going on. Since this is PSTN, all of the
> calls are ulaw, so there should be no need to transcode.  In this
> situation, Asterisk was consuming 72% of the CPU steady.  Since our goal is
> to get well above 80 agents, we realized we had a problem.
>
> I have several questions around this scenario:
>
> 1) I had thought that ConfBridge optimizes media paths where it can. In
> our use-case, the number of participants in ConfBridge will always be
> exactly 1 or 2 - both of which are highly optimizable. Is my understanding
> wrong?
>

Unfortunately, that isn't the case.

While ConfBridge uses aspects of the Bridging Framework, it only uses a
small portion of it - namely the bridge_softmix mixing technology. It
doesn't participate as a member in the 'smart bridge' portion of the
Bridging Framework, and as such, it always views the participants in the
bridge as being in a multi-party bridge.

(From a slightly skewed OO perspective, you can view ConfBridge as a
subclass of a bridge that provides behaviour, while the mixing technologies
as classes that provide the strategy for how media is mixed. Unfortunately,
ConfBridge didn't get fully subclassed in Asterisk 12, hence why it is
"locked" into using the softmix bridge technology; more on that later)

(Also, the smart bridging portion of the Bridging Framework didn't really
get exercised until Asterisk 12. It exists in Asterisk 11, but is unused.)

The softmix mixing technology always performs media mixing on the
participants. This is done for a few reasons:
(1) As a mixing technology, it does one thing and one thing well. Other
mixing technologies are responsible for doing efficient two-party mixing,
and it leaves that job up to them.
(2) ConfBridge was always intended as a conferencing application. The
notion of having one or two participants in it was never really part of the
use case for the application.
(2) Switching between softmix and other mixing technologies would be tricky
for ConfBridge. In order to provide a lot of the features in ConfBridge -
such as silence/voice activity detection - the voice frame format needs to
be in a common format. Since the softmix bridge technology requires frames
to be in a common format, that logic lives in the softmix bridge technology
- ConfBridge merely passes that information out to the user. A lot of the
video handling stuff is also in the softmix mixing technology.

All of that aside, we have an issue in Asterisk 12 (ASTERISK-21271) that
would make ConfBridge act as a more full participant in the Bridging
Framework by properly and fully subclassing it - this would require
implementing the ast_bridge_methods virtual table in ConfBridge in Asterisk
12. We didn't get to it for a number of reasons - mostly because the impact
of ConfBridge acting as a limited participant in the Bridging Framework is
pretty small in Asterisk 12 (unlike in Asterisk 11, unfortunately), and
doing the subclassing is quite a lot of work. More on that below.



>
> 2) Why is ConfBridge eating so much CPU with only 1 channel per bridge? Is
> it transcoding? Is it mixing?
>

Most likely it is. You would at least have ulaw => slin as frames are read
from the channel, and slin => ulaw on the way back out.

Obviously with one participant this isn't ideal, since they're only mixing
to themselves.

It would probably be possible to optimize softmix_bridge_write to look at
the number of channels in the bridge (bridge->channels) and, if there is
only one channel, to drop the media frame and instead write a silence frame
(or something similar) back to the originating channel.

That should be applicable to both 11 and 12.


>
> 3) Is there a way to generally reduce ConfBridge overhead? Can we specify
> the codec up front? All of my calls should be ulaw so there should be no
> need to transcode before mixing.
>

Unfortunately, that doesn't exist right now. Because the softmix bridge
allows a channel with any codec type to join - and doesn't force
participants to only have the same codec - it makes everyone translate
to/from slin.



>
> 4) We are now looking at moving to the AMI Bridge command - should we
> expect better performance as a result?
>

Yes. Bridge in Asterisk 11 would use the features.c bridging loop. While
that wouldn't have multi-party capabilities (which it doesn't appear like
you need since the max participants was limited to 2), it would also defer
to using native bridging when possible, and it certainly shouldn't
transcode in general. Of course, you could Redirect participants into a
ConfBridge when you actually need multi-party, and keep them in the
two-party bridge otherwise.

I mentioned earlier that this was less of a concern in Asterisk 12. It
isn't because ConfBridge became magically better: rather, the rest of the
system got a lot better - which made ConfBridge less useful, particularly
for your scenario.

(1) Bridging in general in Asterisk 12 now uses 'smart bridging'. This
means bridges created via the Bridge dialplan application/AMI action (or
better, through ARI) can transition between native mixing, two-party mixing
in the core, and multi-party mixing via softmix. When possible, bridges
attempt to be as efficient as possible - which means they err on the native
and only choose softmix as a last resort. That would work well in your case.

(2) Bridges can be controlled through ARI. That means you don't have to
move channels between bridges to gain capabilities - you can just add them
to bridges and expect the bridge to pick the right way to mix the media.
That's also why ConfBridge is less useful in this situation: you wouldn't
really need to use ConfBridge (which is a heavy weight dialplan
application) to create a multi-party bridge: you'd just add channels to the
bridge and let them get multi-mixed.

Hope this helps. There's probably a dozen more questions you might have
after all this - I'm going to try and get the doxygen in Asterisk updated
with a better description of the bridging architecture over the Christmas
break.

Matt

-- 
Matthew Jordan
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-app-dev/attachments/20131224/7561b655/attachment.html>