[asterisk-users] TE121 - Idle system load at ~0.3 - Bad DAHDI 2.2.0.2 behaviour ?!

[Shaun Ruffell]

On 11/16/2009 09:42 AM, Ex Vito wrote:
>   Shaun,
>
>   Thanks for your feedback. See my inline comments.
>
>
> On Fri, Nov 13, 2009 at 7:18 PM, Shaun Ruffell<sruffell at digium.com>  wrote:
>>
>> It appears there may be a regression in dahdi-linux 2.2.0 with regards to
>> the wcte12xp driver and the VPMADT032 module (as discussed
>> https://issues.asterisk.org/view.php?id=15724).  Would you be willing to try
>> at least revision 7584 of
>> http://svn.asterisk.org/svn/dahdi/linux/branches/2.2 and report your results
>> on that issue?
>>
>
>   We will, either on 15724 or on 15798, both, I'd say, closely related
>   to what we're experiencing.
>
>   First, however, we will test 2.1.0.4 for at least 10 days to
>   effectively confirm are experiencing the regression you refer. We
>   defined 10 stable days because we've been experiencing 1 - 3
>   failures per week with 2.2.0.2.
>
>   So, we will move to 2.1.0.4 later today and, if there are no incidents,
>   we will move to SVN revision 7584 or later on Nov 27th.
>
>
>>
>> The idle load you're seeing can be a little misleading, but essentially,
>> once you load the drivers for both the wctdm24xxp and wcte12xp, there is a
>> fixed cost associated with continuously moving the TDM data to/from the
>> card.  The load imposed by the drivers would only go up after this point if
>> a) software echocan is enabled, or b) you're conferencing many calls in the
>> kernel.  Otherwise....it's fixed.
>>
>
>   I agree the load can be misleading however, consider:
>
>   - The system is really idle - zero calls, zero activity, no other software.
>   - 0.3 to 0.4 load on a modern CPU is a lot of processing, is there
>     that much data to move around ?! :)

yes :)  All the TDM data is still moved between the host and the card as 
if there were 24 calls up.  The driver doesn't make any distinction 
between calls being up or calls not being up unless you're using 
software echocan.

>
>   We tested some variations and here is what we found:
>   (recall, AEX410 with no VPM is physically installed)
>
>   DAHDI 2.2.0.2
>   - Removed TE121 - Idle load is 0
>   - TE121 without VPM - Idle load is 0.3 - 0.4
>   - TE121 with VPM - Idle load is 0.3 - 0.4
>
>   DAHDI 2.1.0.4
>   - TE121 without VPM - Idle load is 0.01 - 0.02
>   - TE121 with VPM - Idle load is 0.01 - 0.05
>
>   DAHDI 2.2.0.1
>   - TE121 with VPM - Idle load is 0.26 - 0.32
>
>   DAHDI 2.2.0
>   - TE121 with VPM - Idle load is 0.31 - 0.36
>
>   DAHDI SVN-branch-2.2-r7584
>   - TE121 with VPM - Idle load is 0.69 - 0.82
>
>   Under all cases, we stopped asterisk, unloaded DAHDI,
>   rebuilt DAHDI (+Asterisk if needed), loaded DAHDI, started
>   Asterisk, waited 120s, tested one inbound call + one outbound
>   call.
>
>   In short:
>
>   - DAHDI 2.1, Idle load is 0
>   - DAHDI 2.2, Idle load is 0.3 - 0.4
>   - DAHDI 2.2. SVN, Idle load is roughly twice as in 2.2 releases
>
>   So, while your explanation regarding system load makes sense,
>   I find it odd that 2.2, which apparently is not working ok for us
>   in this case, is showing such behaviour when 2.1 does not and
>   neither Zaptel 1.2/1.4 used many other systems I've installed.
>
>   What would you say the explanation for these observations is ?

The change from 2.1 to 2.2 I believe is a timing anomaly.  The work 
related to checking the alarm conditions from the framer on the TE122 
was moved from the interrupt handler and into a workqueue, and therefore 
the system is more accurately able to account for it's time.  Between 
2.2.0.2 and the current head of the 2.2 branch, the frequency that that 
timers are checked was doubled (from once every 200ms to once every 
100ms) and that is a likely candidate for the doubling of the load. 
I'll need to run my own tests to confirm this.

>
>   Would you say that there is a correlation between the load and
>   the suspected not so good behaviour we're getting from DAHDI
>   2.2.0.2 on our case ?
>
>   Or is it just a coincidence ?

My belief is that this is just a coincidence.  At least from the driver 
standpoint, if you're not receiving any "latency increases" or IRQ 
misses isn't constantly rolling up during normal operation, then the 
data is consistent coming off the card.  There can still be problems 
moving that data up to user space if there is something that is 
preventing asterisk from being scheduled in a timely enough fashion to 
keep the kernel buffers non-empty (for transmit) or non-full (for receive).

-- 
Shaun Ruffell
Digium, Inc. | Linux Kernel Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: www.digium.com & www.asterisk.org