[asterisk-bugs] [JIRA] (ASTERISK-28893) pbx_realtime: Cascading deadlock due to ast_autoservice_stop blocking

[Joshua C. Colp (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=250770#comment-250770 ] 

Joshua C. Colp commented on ASTERISK-28893:
-------------------------------------------

I analyzed the backtrace. A channel is deadlocked while trying to stop autoservice as a result of pbx_realtime being in use, same as in ASTERISK-21228. Other threads are iterating through the channel container which requires locking the channel that is deadlocked, they then become deadlocked as they wait for the lock. If pbx_realtime were not in use, autoservice would not be in use, the code path would not be executed, the deadlock would not occur.

> pbx_realtime: Cascading deadlock due to ast_autoservice_stop blocking
> ---------------------------------------------------------------------
>
>                 Key: ASTERISK-28893
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28893
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Channels/chan_sip/General, PBX/pbx_realtime
>    Affects Versions: 13.31.0
>         Environment: Centos 7.7.1908
> Linux 3.10.0-1062.9.1.el7.x86_64
>            Reporter: Martin Nyström
>         Attachments: dumps.tar.gz
>
>
> We seem to repeatable run into full udp buffers in our Asterisk. I am trying to figure out if Asterisk is doing something incorrectly or if the issue lays in the OS level. When this happens, as you can imagine, all UDP traffic starts to get dropped and an asterisk restart is required.
> Now I could increase the UDP buffers, but it feels like that will only push the issue to the future instead and if its in Asterisk then that solution would not be ideal or long-term.
> [root at server ~]# netstat -c --udp -an | grep 5060
> udp   213248      0 0.0.0.0:5060            0.0.0.0:*                          
> udp   213248      0 0.0.0.0:5060            0.0.0.0:*                          
> udp   213248      0 0.0.0.0:5060            0.0.0.0:*                          
> udp   213248      0 0.0.0.0:5060            0.0.0.0:*                          
> udp   213248      0 0.0.0.0:5060            0.0.0.0:*                          
> udp   213248      0 0.0.0.0:5060            0.0.0.0:*   
> [root at server ~]# sysctl -a | grep mem
> net.core.optmem_max = 20480
> net.core.rmem_default = 212992
> net.core.rmem_max = 212992
> net.core.wmem_default = 212992
> net.core.wmem_max = 212992
> net.ipv4.igmp_max_memberships = 20
> net.ipv4.tcp_mem = 1443162	1924218	2886324
> net.ipv4.tcp_rmem = 4096	87380	6291456
> net.ipv4.tcp_wmem = 4096	16384	4194304
> net.ipv4.udp_mem = 1445055	1926743	2890110
> net.ipv4.udp_rmem_min = 4096
> net.ipv4.udp_wmem_min = 4096
> sysctl: reading key "net.ipv6.conf.all.stable_secret"
> sysctl: reading key "net.ipv6.conf.default.stable_secret"
> sysctl: reading key "net.ipv6.conf.ens192.stable_secret"
> sysctl: reading key "net.ipv6.conf.ens224.stable_secret"
> sysctl: reading key "net.ipv6.conf.lo.stable_secret"
> vm.lowmem_reserve_ratio = 256	256	32
> vm.memory_failure_early_kill = 0
> vm.memory_failure_recovery = 1
> vm.nr_hugepages_mempolicy = 0
> vm.overcommit_memory = 1
> Attached core dumps when this happens.



--
This message was sent by Atlassian JIRA
(v6.2#6252)