[asterisk-users] PJSIP Lockup

Mon Apr 6 11:36:06 CDT 2020

On Thu, Apr 2, 2020 at 11:34 AM Nick Olsen <nick at floridavirtualsolutions.com>
wrote:

> Paddy, It's pretty easy to spot from the CLI.
>
> A voicemail gets called. And the screen basically stops scrolling from
> there. Eventually you'll get the "Task processors exceeded 500 queued
> tasks" or something like that. And maybe channels attempting to hangup due
> to lack of RTP (If you have no-rtp timers configured).
>
> Once you find the problem mailbox, You can call it via any method and
> it'll deadlock every time as soon as it tries to play the mailboxes unavail
> greeting. I've never had it occur when there is no unavail greeting. Each
> case deleting the problem recording from the database fixes the issue, And
> subsequent recordings for the same mailbox have no issue.
>

Given that the issue appears to be related to specific rows and not the
database in general, you might want to get a backtrace while the system is
locked as Josh suggested earlier.   Once you get the backtraces, open an
issue ar https://issues.asterisk.org.

https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace
NOTE: you do NOT need to recompile with the DEBUG_THREADS, MALLOC_DEBUG,
DONT_OPTIMIZE or BETTER_BACKTRACES but the Asterisk binaries need to still
have the symbols in them (un-stripped).

>
> *Nick Olsen*
> Network Engineer
> Office: 321-408-5000 x103
> Mobile: 321-794-0763
>
>
>
> On Wed, Apr 1, 2020 at 9:04 PM Paddy Grice <paddy at wizaner.com> wrote:
>
>> Hi All
>>
>> This sounds just like a problem I have had and still investigating having
>> moved to 16.9 using chan_sip. I am still trying to repeat the problem it
>> looks from debug that the issue is either voicemail of call transfer but I
>> cant consistently repeat it.
>>
>> Voicemail is using ODBC and I just imported the data from the old system
>> into the new database.
>>
>> Nick - if you have any more info I would be grateful
>>
>> TIA
>>
>> Paddy
>>
>> ------------------------------
>> *From:* asterisk-users [mailto:asterisk-users-bounces at lists.digium.com] *On
>> Behalf Of *Nick Olsen
>> *Sent:* 01 April 2020 18:54
>> *To:* Asterisk Users Mailing List - Non-Commercial Discussion
>> *Subject:* Re: [asterisk-users] PJSIP Lockup
>>
>> We ultimately found this to be a voicemail issue. The voicemail is held
>> in MYSQL as well (via ODBC). And we found when attempting to playback a
>> customers voicemail unavail greeting is when the deadlock would occur
>> (Immediately, every time. Throwing the same "task processors" errors, And
>> making pjsip completely unresponsive). We had imported a number of
>> greetings from a legacy asterisk system and the vast majority of them
>> worked. When we deleted the row containing the customers unavail greeting
>> (making asterisk revert to read the mailbox number) all issues went away.
>> If we re-record the customers unavail greeting it works fine and the
>> problem doesn't reoccur. This was one out of ~250 voicemails imported.
>>
>> Since then we've done a few more migrations and they've all gone smooth
>> with the exception of the most recent one. ~50% of the imported greetings
>> have caused asterisk to deadlock. We've been checking them now at time of
>> migration.
>>
>> What I can't figure out is what it doesn't like about the greeting. It
>> was on a previous asterisk system working fine. The row looks identical to
>> a working one. The only thing I can guess is something about the blob for
>> the recording goes wrong. It would be nice if asterisk handled that more
>> gracefully.
>>
>> I post this mostly just for internet history. To hopefully help the next
>> guy out who has this same issue.
>>
>> *Nick Olsen*
>> Network Engineer
>> Office: 321-408-5000 x103
>> Mobile: 321-794-0763
>>
>>
>>
>> On Mon, Mar 2, 2020 at 8:29 PM Joshua C. Colp <jcolp at sangoma.com> wrote:
>>
>>> On Mon, Mar 2, 2020 at 4:24 PM Nick Olsen <
>>> nick at floridavirtualsolutions.com> wrote:
>>>
>>>> Thanks for the info, Joshua.
>>>>
>>>> Does PJSIP handle database access the same way Chan_sip did? We had a
>>>> number of boxes running chan_sip referencing the same mysql server without
>>>> issue.
>>>>
>>>> We're going to attempt to get a backtrace on the next occurance. We're
>>>> also going to run a local copy of the database on the same physical
>>>> asterisk instance and have the system reference it. Just to "throw
>>>> everything at the wall".
>>>>
>>>
>>> It uses the same underlying API and layer. It can do more frequent
>>> database access though due to queries and because PJSIP is multithreaded.
>>>
>>> --
>>> Joshua C. Colp
>>> Asterisk Technical Lead
>>> Sangoma Technologies
>>> Check us out at www.sangoma.com and www.asterisk.org
>>> --
>>> _____________________________________________________________________
>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>
>>> Check out the new Asterisk community forum at:
>>> https://community.asterisk.org/
>>>
>>> New to Asterisk? Start here:
>>>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>>>
>>> asterisk-users mailing list
>>> To UNSUBSCRIBE or update options visit:
>>>    http://lists.digium.com/mailman/listinfo/asterisk-users
>>
>> --
>> _____________________________________________________________________
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> Check out the new Asterisk community forum at:
>> https://community.asterisk.org/
>>
>> New to Asterisk? Start here:
>>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>>
>> asterisk-users mailing list
>> To UNSUBSCRIBE or update options visit:
>>    http://lists.digium.com/mailman/listinfo/asterisk-users
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at:
> https://community.asterisk.org/
>
> New to Asterisk? Start here:
>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users

-- 
George Joseph
Asterisk Software Developer
direct/fax +1 256 428 6012
Check us out at www.sangoma.com and www.asterisk.org
[image: image.png]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20200406/c602224c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 5142 bytes
Desc: not available
URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20200406/c602224c/attachment.png>