[asterisk-bugs] [JIRA] (ASTERISK-28997) res_pjsip: Asterisk locks up and stops processing any SIP requests

[Yury Kirsanov (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=251576#comment-251576 ] 

Yury Kirsanov edited comment on ASTERISK-28997 at 7/31/20 5:14 AM:
-------------------------------------------------------------------

Hi Joshua,
This was occurring every day or so and yes, we are doing a lot of reloads each day, basically, this Asterisk server has a custom GUI that allows customers to create their own dial plans and apply changes by reloading Asterisk with new configs. We have about 8 such servers and all the rest are working fine. On this server it happened to be a coincidence - we had issues with Asterisk segfaulting due to T.38 fax, this was happening only to this server as one of the customers used fax a lot. It turned out to be a bug and we had it fixed by Asterisk team in ticket 28923. But before that patch one of our admins was trying to do everything possible to fix the issue so he has taken an example for Asterisk sorcery.conf file from official Asterisk tuning manual (https://wiki.asterisk.org/wiki/display/AST/Performance+Tuning) and applied it only to this server. That didn't help with segfaults but after I have applied patch 28923 configuration of sorcery.conf stayed on that server, of which I wasn't aware. Since that time segfaults have gone away but we started to experience lockups. Now after I have removed that stasis.conf server behaves perfectly fine. I can confirm that we didn't have any lock-ups since 22nd of July. Thanks a lot for your help!


was (Author: lt_flash):
Hi Joshua,
This was occurring every day or so and yes, we are doing a lot of reloads each day, basically, this Asterisk server has a custom GUI that allows customers to create there own dial plans and apply changes by reloading Asterisk with new configs. We have about 8 such servers and all the rest are working fine. On this server it happened to be a coincidence - we had issues with Asterisk segfaulting due to T.38 fax, this was happening only to this server as one of the customers used fax a lot. It turned out to be a bug and we had it fixed by Asterisk team in ticket 28923. But before that patch one of our admins was trying to do everything possible to fix the issue so he has taken an example for Asterisk sorcery.conf file from official Asterisk tuning manual (https://wiki.asterisk.org/wiki/display/AST/Performance+Tuning) and applied it only to this server. That didn't help with segfaults but after I have applied patch 28923 configuration of sorcery.conf stayed on that server, of which I wasn't aware. Since that time segfaults have gone away but we started to experience lockups. Now after I have removed that stasis.conf server behaves perfectly fine. I can confirm that we didn't have any lock-ups since 22nd of July. Thanks a lot for your help!

> res_pjsip: Asterisk locks up and stops processing any SIP requests
> ------------------------------------------------------------------
>
>                 Key: ASTERISK-28997
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28997
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Resources/res_pjsip, Resources/res_sorcery_memory_cache
>    Affects Versions: 16.11.1
>         Environment: Ubuntu Linux 18.04.4 LTS, kernel 4.15.0-111-generic
>            Reporter: Yury Kirsanov
>            Assignee: Unassigned
>            Severity: Minor
>              Labels: fax
>         Attachments: backtrace-threads.txt, taskprocessors.txt, threads.txt
>
>
> Hi,
> We have a problem with one of many of our Asterisk-based servers. From time to time it just locks up and stops processing SIP messages, complaining about pjsip/distributor queue has reached 500 scheduled tasks. We have several servers based on same Asterisk version with exactly the same environment, but different Asterisk configuration files, some have more SIP devices and more complex dial plans, some has less.
> This issue happens at any time of the day and I don't see any way to reproduce it. There is no segfault generated as Asterisk keeps running, the only thing is that all distributor queues are overflown for some reason.
> After Asterisk restart it starts to work like nothing happened. I've checked all network interfaces and there is no DDoS attack or anything like that.
> Our Asterisk is patched with ASTERISK-28923 patch and it's the same server that was segfaulting before. Other similar servers are running unpatched version of Asterisk 16.11.1.
> I'm attaching debug logs that I was able to grab during this lockup. Asterisk was compiled with 'NO_DEBUG' option.
> Please let me know if I can provide any additional information or recompile Asterisk with some additional debug versions. Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)