[asterisk-users] Bridged PRI calls - processor involvement?

[Steve Hanselman]

Tracked this down (or more to the point found the issue causing it), it
was high levels of bursty disk activity.

The iowait went through the roof (30-40%).

The disks are scsi serviced by an MPT-Fusion controller in a Dell
Poweredge 2850.

We're using LVM to bind the disks into a JBOD set.

Steve

-----Original Message-----
From: asterisk-users-bounces at lists.digium.com
[mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Steve
Hanselman
Sent: 11 June 2007 10:40
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: RE: [asterisk-users] Bridged PRI calls - processor involvement?

I checked for BIOS upgrades the other week and there were none.

I'm starting to suspect kernel changes as being the reason for this so I
guess I'm going to have to remove some of the patchy disk activity to
smooth the load and then start researching!!!

Steve


-----Original Message-----
From: asterisk-users-bounces at lists.digium.com
[mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Gordon
Henderson
Sent: 11 June 2007 09:53
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: RE: [asterisk-users] Bridged PRI calls - processor involvement?

On Mon, 11 Jun 2007, Steve Hanselman wrote:

> This is the io wait figure from vmstat.
>
> If I run a vmstat 2 whilst I'm on a call I can see that the "wa"
figure
> gets very high when the missing audio problem occurs.

I once looked after a Dell 2850 that exhibited some odd behaviour that I

never got to the bottom of. It would seem to lock-up or just crawl for
2-3 
seconds every now & then. Nothing logged, noting on the console. It had
6 
SCSI drives fitted. I rebuilt the server twice, rebuilt the s/w RAID 
arrays twice, even put all 6 drives in another box (which appeared
towork 
OK), but never got to the bottom of it. Each disk would benchmark really

fast individually, Ethernet performance was good, but overall, when 
everything was used together, it just didn't feel right. (compared to 
other Dells and other servers, biger & smaller that I've built and used 
over the years). I'd see processes hung in a "D" state (waiting for IO
to 
complete) for what seemed like an overly long time, (waiting on disk),
but 
...

I suspected a BIOS pproblem, but never had a chance to get to the bottom

of it. (It was a live server doing *everything* for a small company -
DNS, 
NIS, NFS, Intranet/WiKi, Samba, etc, etc, etc,... so taking it offline
for 
tests was problematic)

So I wonder if looking at the BIOS and seeing if there are any Dell 
upgrades avalable for it might help?

Gordon


  >
> Steve
>
>
> -----Original Message-----
> From: asterisk-users-bounces at lists.digium.com
> [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Matthew
> Fredrickson
> Sent: 08 June 2007 19:38
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [asterisk-users] Bridged PRI calls - processor
involvement?
>
> iowait time?  I'm not familiar with that.  Where are you seeing that?
> Also, is it a reproducible problem?
>
> ---
> Matthew Fredrickson
> Software Engineer
> Digium, Inc.
>
> On Jun 8, 2007, at 11:23 AM, Steve Hanselman wrote:
>
>> It probably did but we run in updates every week and nobody can state
>> exactly when the problem started only "a few weeks ago" - not very
>> helpful.
>>
>> I can see that when I hear the issue the iowait time is high on the
>> processor.
>>
>> Steve
>> -----Original Message-----
>> From: asterisk-users-bounces at lists.digium.com
>> [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Matthew
>> Fredrickson
>> Sent: 08 June 2007 15:43
>> To: Asterisk Users Mailing List - Non-Commercial Discussion
>> Subject: Re: [asterisk-users] Bridged PRI calls - processor
>> involvement?
>>
>> Did it accompany an update you made?  If you can find out what
version
>> the problem started occurring, that would help in fixing the problem,
>>
>> Matthew Fredrickson
>> Software/Hardware Engineer
>> Digium, Inc.
>>
>> On Jun 8, 2007, at 2:59 AM, Steve Hanselman wrote:
>>
>>> The setup.
>>>
>>> Asterisk is on a 3G Zeon Dell 2850 running Fedora Core 5/6 (all yum
>>> updates applied), the TE410 lives on it's own interrupt.
>>> Asterisk sits between our telco and a PRI enabled PBX.
>>> These are the relevant versions installed:
>>>
>>> Linux: 2.6.20-1.2316.fc5smp
>>> Zaptel: 1:1.4.2.1-34.fc5
>>> Asterisk: 1:1.4.0-34.fc5.at
>>> Libpri: 1:1.4.0-16.fc5.at
>>> Wildcard details:
>>> Found TE4XXP at base address fe3ffc00, remapped to f88bec00
>>> TE4XXP version c01a016a, burst OFF, slip debug: OFF
>>> Octasic optimized!
>>> FALC version: 00000005, Board ID: 00
>>> Reg 0: 0x377bb400
>>> Reg 1: 0x377bb000
>>> Reg 2: 0xffffffff
>>> Reg 3: 0x00000000
>>> Reg 4: 0x00000001
>>> Reg 5: 0x00000000
>>> Reg 6: 0xc01a016a
>>> Reg 7: 0x00001f00
>>> Reg 8: 0x00000000
>>> Reg 9: 0x00ff0000
>>> Reg 10: 0x0000004a
>>> TTE4XXP: Launching card: 0
>>> TE4XXP: Setting up global serial parameters
>>> Found a Wildcard: Wildcard TE410P (3rd Gen)
>>> TE4XXP: Span 1 configured for CCS/HDB3/CRC4
>>> TE4XXP: Span 2 configured for CCS/HDB3/CRC4
>>>
>>>
>>>
>>> The problem:
>>>
>>> At random points during calls we lose 1-3 seconds of speech (both
> ways
>>> both callee and caller), this can be replicated (or at least a very
>>> good
>>> approximation!) by generating a high level of interrupt/cpu activity
>>> (for instance copying data from a USB caddy as we tried the other
day
>>> in
>>> an attempt to reproduce this more reliably).
>>>
>>> The calls are bridged PRI:PRI calls, no VOIP involvement.
>>>
>>> This was not a problem until approx 3-4 weeks ago, but I can't tie
it
>>> down to an exact date.
>>>
>>> Steve
>>>
>>>
>>>> Interrupt sharing is not a problem anymore with those cards.  What
>>>> version of zaptel did you try installing?  Can you explain more
> about
>>>> your problems?  Also, your configuration and setup would help out
as
>>>> well.
>>>>
>>>> ---
>>>> Matthew Fredrickson
>>>> Digium, Inc.
>>>
>>>
>>> The information contained in this email is intended for the personal
>>> and confidential use
>>> of the addressee only. It may also be privileged information. If you
>>> are not the intended
>>> recipient then you are hereby notified that you have received this
>>> document in error and
>>> that any review, distribution or copying of this document is
strictly
>>> prohibited. If you have
>>> received  this communication in error, please notify Brendata
>>> immediately on:
>>>
>>> +44 (0)1268 466100, or email 'technical at brendata.co.uk'
>>>
>>> Brendata (UK) Ltd
>>> Nevendon Hall, Nevendon Road, Basildon, Essex. SS13 1BX  UK
>>> Registered Office as above. Registered in England No. 2764339
>>>
>>> See our current vacancies at www.brendata.co.uk
>>> _______________________________________________
>>> --Bandwidth and Colocation provided by Easynews.com --
>>>
>>> asterisk-users mailing list
>>> To UNSUBSCRIBE or update options visit:
>>>    http://lists.digium.com/mailman/listinfo/asterisk-users
>>
>> _______________________________________________
>> --Bandwidth and Colocation provided by Easynews.com --
>>
>> asterisk-users mailing list
>> To UNSUBSCRIBE or update options visit:
>>    http://lists.digium.com/mailman/listinfo/asterisk-users
>>
>>
>> The information contained in this email is intended for the personal
>> and confidential use
>> of the addressee only. It may also be privileged information. If you
>> are not the intended
>> recipient then you are hereby notified that you have received this
>> document in error and
>> that any review, distribution or copying of this document is strictly
>> prohibited. If you have
>> received  this communication in error, please notify Brendata
>> immediately on:
>>
>> +44 (0)1268 466100, or email 'technical at brendata.co.uk'
>>
>> Brendata (UK) Ltd
>> Nevendon Hall, Nevendon Road, Basildon, Essex. SS13 1BX  UK
>> Registered Office as above. Registered in England No. 2764339
>>
>> See our current vacancies at www.brendata.co.uk
>> _______________________________________________
>> --Bandwidth and Colocation provided by Easynews.com --
>>
>> asterisk-users mailing list
>> To UNSUBSCRIBE or update options visit:
>>    http://lists.digium.com/mailman/listinfo/asterisk-users
>
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>
>
> The information contained in this email is intended for the personal
and confidential use
> of the addressee only. It may also be privileged information. If you
are not the intended
> recipient then you are hereby notified that you have received this
document in error and
> that any review, distribution or copying of this document is strictly
prohibited. If you have
> received  this communication in error, please notify Brendata
immediately on:
>
> +44 (0)1268 466100, or email 'technical at brendata.co.uk'
>
> Brendata (UK) Ltd
> Nevendon Hall, Nevendon Road, Basildon, Essex. SS13 1BX  UK
> Registered Office as above. Registered in England No. 2764339
>
> See our current vacancies at www.brendata.co.uk
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users


The information contained in this email is intended for the personal and
confidential use
of the addressee only. It may also be privileged information. If you are
not the intended
recipient then you are hereby notified that you have received this
document in error and
that any review, distribution or copying of this document is strictly
prohibited. If you have 
received  this communication in error, please notify Brendata
immediately on: 

+44 (0)1268 466100, or email 'technical at brendata.co.uk' 

Brendata (UK) Ltd
Nevendon Hall, Nevendon Road, Basildon, Essex. SS13 1BX  UK
Registered Office as above. Registered in England No. 2764339

See our current vacancies at www.brendata.co.uk
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users


The information contained in this email is intended for the personal and confidential use
of the addressee only. It may also be privileged information. If you are not the intended
recipient then you are hereby notified that you have received this document in error and
that any review, distribution or copying of this document is strictly prohibited. If you have 
received  this communication in error, please notify Brendata immediately on: 

+44 (0)1268 466100, or email 'technical at brendata.co.uk' 

Brendata (UK) Ltd
Nevendon Hall, Nevendon Road, Basildon, Essex. SS13 1BX  UK
Registered Office as above. Registered in England No. 2764339

See our current vacancies at www.brendata.co.uk