[Asterisk-Users] wcfxs causing constant CPU spikes - PATCHING LINUX FOR LOW LATENCY

Thu Dec 16 00:40:52 MST 2004

> -----Original Message-----
> From: Eric Bishop [mailto:asterisk.eric at gmail.com] 
> Sent: December 15, 2004 7:53 PM
> To: jim at vanmeggelen.ca
> Subject: Re: [Asterisk-Users] wcfxs causing constant CPU spikes
> 
> 
> Jim,
> 
> As I mentioned in my email, I have already tried a fresh 
> kernel an get the same result. 

You did say that. My apologies, I missed it.

> I would love to test out 
> whatever solution you may have......

I have a few things percolating.

First, a little introduction for anyone who isn't sure what's going on:

One of the big challenges that a Digium TDM card has is getting the
attention it needs from the processor. These cards do not have a DSP on
board, and thus are totally dependant on the CPU to handle this for
them. They acheive this by establishing a constant connection to the
CPU, a kind of synchronous channel kept alive by their repeated IRQing
(this isn't technically correct, but it serves as an analogy). Because
of the nature of a media stream (audio, video, whatever), any latency
will be experienced by the user as a quality issue, so it is essential
that the TDM card always receive priority service from the scheduler. 

Here's a fascinating article on managing and adjusting latency. Check
out the lspci command. Try setting latency timing between 00 and FF,
tell me what (if anything) happens.
http://www-106.ibm.com/developerworks/library/l-hw2.html 
(this article was written by Daniel Robbins, who created Gentoo)

The needs of these cards is very similar in nature to what is required
from a Digital Audio Workstation (DAW). DAWs are used as software-based
audio editing studios, and if any DSP work needs to be done in the CPU,
the same latency problems can exist.

What I did was track down the mailing list that the linux audio
developers use (http://www.linuxdj.com/audio/lad/), in the hopes that
not only would they have experienced similar performance problems as us,
but also have some solutions.

Well, they DO have the same problems that we do, and, more importantly,
there have been kernel patches developed (not just one, but at least
three different ones) which are specifically intended to address IRQ
latency.

Here are some of the resources I used. They discuss problems that are
eerily similar to ours:

http://www.linuxdevices.com/files/article027/rh-rtpaper.pdf
http://www.linuxdj.com/audio/quality/#dropouts
http://ardour.org/requirements.html

Here are the patches:
First, Andrew Morton's:
http://www.zip.com.au/~akpm/linux/schedlat.html

Next, Robert Love's patches
http://www.kernel.org/pub/linux/kernel/people/rml/preempt-kernel/v2.4/

Finally, Ingo Molnar's controversial, but high-performance patches
http://people.redhat.com/mingo/lowlatency-patches/

And for those who need some tips on kernel rebuilding:
http://www.digitalhermit.com/linux/Kernel-Build-HOWTO.html#INTRO

If we can get a few people to test these out, we can determine if the
Digium cards and Asterisk would benefit from the same tweaks as the
audio engineering folks use.

Thanks.

> Thanks...
> 
> 
> On Wed, 15 Dec 2004 19:25:46 -0500, Jim Van Meggelen 
> <jim at vanmeggelen.ca> wrote:
> > First of all, from what I've read, you need to get rid of the stock 
> > Fedora kernel -- RedHat's kernels seem to get a lot of negative 
> > attention these days. Download a fresh one from www.kernel.org, and 
> > re-complile. Here's some decent instructions: 
> > http://www.digitalhermit.com/linux/Kernel-Build-HOWTO.html#INTRO
> > 
> > There's more to this than that, but my advice would be to not even 
> > bother until you've built a fresh kernel over the RedHat 
> one. (check 
> > out the list archives -- you're not the only one with this problem).
> > 
> > I'm right in the middle of something that may have a huge impact on 
> > all the IRQ latency issues everyone's been having. Let me know if 
> > you're interested in using your system as a guinea pig.
> > 
> > Cheers,
> > 
> > Jim.
> > 
> > 
> > > -----Original Message-----
> > > From: asterisk-users-bounces at lists.digium.com
> > > [mailto:asterisk-users-bounces at lists.digium.com] On 
> Behalf Of Eric 
> > > Bishop
> > > Sent: December 15, 2004 5:37 PM
> > > To: asterisk-users at lists.digium.com
> > > Subject: [Asterisk-Users] wcfxs causing constant CPU spikes
> > >
> > >
> > > Hi All,
> > >
> > > I have a problem (at least I think it's a problem) where 
> the wcfxs 
> > > module causes constant CPU usage spikes. The card being used is a 
> > > Digium Wildcard TDM400P with 3 FXO modules and 1 FXS (TDM31B).
> > >
> > > Monitoring my otherwise idle asterisk box (with top) I see once 
> > > every 3-5 seconds hi (hardware interrupt) spike to 
> 30%-40% and then 
> > > drop back to 0 again. This is the case wtih both asterisk 
> stable and 
> > > CVS HEAD. I am running Fedora Core 2 (also tried with Fedora Core 
> > > 3). I have also tried installing a vanilla kernel as suggested 
> > > previously on the list, all with the same result. Unloading the 
> > > wcfxs module fixes the problem.
> > >
> > > Is this normal for the wcfxs module or am I being paranoid?
> > > _______________________________________________
> > > Asterisk-Users mailing list
> > > Asterisk-Users at lists.digium.com
> > > http://lists.digium.com/mailman/listinfo/> asterisk-users
> > > To
> > > UNSUBSCRIBE or update options visit:
> > >
> > http://lists.digium.com/mailman/listinfo/asterisk-users
> > 
> >
> 