[asterisk-bugs] [JIRA] (ASTERISK-26831) res_rtp_asterisk: Race condition when RTCP and WebRTC is used
Badalian Vyacheslav (JIRA)
noreply at issues.asterisk.org
Thu Apr 20 09:13:58 CDT 2017
[ https://issues.asterisk.org/jira/browse/ASTERISK-26831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236618#comment-236618 ]
Badalian Vyacheslav edited comment on ASTERISK-26831 at 4/20/17 9:13 AM:
-------------------------------------------------------------------------
Compiled patch 7
{code}
git fetch https://gerrit.asterisk.org/asterisk refs/changes/43/5343/7 && git checkout FETCH_HEAD
{code}
1. Have deadlocked channels
{code}
ds-asterisk05*CLI> core show channels
Channel Location State Application(Data)
SIP/ndmargaryan_wof- (None) Down (None)
SIP/ivmakevit23393_w (None) Down (None)
SIP/ivmakevit23393_w (None) Down (None)
SIP/ndmargaryan_wof- (None) Down (None)
SIP/ndmargaryan_wof- (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/raburenkov_wof-0 (None) Down (None)
SIP/raburenkov_wof-0 (None) Down (None)
SIP/vyumex_wof-00003 (None) Down (None)
SIP/yuadomracheva_wo (None) Down (None)
SIP/1_stub-000035df (None) Down (None)
SIP/1_stub-000035de (None) Down (None)
SIP/1_stub-000035dd (None) Down (None)
SIP/1_stub-000035dc (None) Down (None)
SIP/1_stub-000035db (None) Down (None)
SIP/1_stub-000035da (None) Down (None)
SIP/1_stub-000035cd (None) Down (None)
SIP/1_stub-000035cc (None) Down (None)
SIP/1_stub-000035cb (None) Down (None)
SIP/1_stub-000035e0 (None) Down (None)
SIP/1_stub-000035d9 (None) Down (None)
SIP/1_stub-000035d8 (None) Down (None)
SIP/1_stub-000035d7 (None) Down (None)
SIP/1_stub-000035d6 (None) Down (None)
SIP/1_stub-000035d5 (None) Down (None)
SIP/1_stub-000035d4 (None) Down (None)
SIP/1_stub-000035d2 (None) Down (None)
SIP/1_stub-000035c9 (None) Down (None)
SIP/iadavidenko_wof- (None) Down AppDial2((Outgoing Line))
SIP/ndmargaryan_wof- (None) Down (None)
48 active channels
0 active calls
6798 calls processed
ds-asterisk05*CLI>
{code}
2. Have errors in logs
{code}
Apr 19 14:12:59 ds-asterisk05.tcsbank.ru asterisk[11220]: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x60b00005b588 (0) at line 122 in stasis_message_create_full of stasis_message.c
Apr 19 14:12:59 ds-asterisk05.tcsbank.ru asterisk[11220]: [2017-04-19 14:12:59] ERROR[2257][C-00000518]: stasis_message.c:122 stasis_message_create_full: Excessive refcount 100000 reached on ao2 object 0x60b00005b588
Apr 19 14:12:59 ds-asterisk05.tcsbank.ru asterisk[11220]: [2017-04-19 14:12:59] ERROR[2257][C-00000518]: stasis_message.c:122 stasis_message_create_full: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x60b00005b588 (0)
{code}
was (Author: slavon):
Compiled patch 7
{code}
git fetch https://gerrit.asterisk.org/asterisk refs/changes/43/5343/7 && git checkout FETCH_HEAD
{code}
1. Have crashes but ASAN don't detect why (no asan logs for crash).
{code}
[36676180.534815] asterisk[11161]: segfault at 0 ip (null) sp 00007f7cb75d2908 error 14 in asterisk[400000+2a0000]
[37288407.540077] asterisk[23859]: segfault at 7f99bc180180 ip 00007f99bc180180 sp 00007f96b08ef638 error 15
{code}
2. Have deadlocked channels
{code}
ds-asterisk05*CLI> core show channels
Channel Location State Application(Data)
SIP/ndmargaryan_wof- (None) Down (None)
SIP/ivmakevit23393_w (None) Down (None)
SIP/ivmakevit23393_w (None) Down (None)
SIP/ndmargaryan_wof- (None) Down (None)
SIP/ndmargaryan_wof- (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/altumanyan_wof-0 (None) Down (None)
SIP/raburenkov_wof-0 (None) Down (None)
SIP/raburenkov_wof-0 (None) Down (None)
SIP/vyumex_wof-00003 (None) Down (None)
SIP/yuadomracheva_wo (None) Down (None)
SIP/1_stub-000035df (None) Down (None)
SIP/1_stub-000035de (None) Down (None)
SIP/1_stub-000035dd (None) Down (None)
SIP/1_stub-000035dc (None) Down (None)
SIP/1_stub-000035db (None) Down (None)
SIP/1_stub-000035da (None) Down (None)
SIP/1_stub-000035cd (None) Down (None)
SIP/1_stub-000035cc (None) Down (None)
SIP/1_stub-000035cb (None) Down (None)
SIP/1_stub-000035e0 (None) Down (None)
SIP/1_stub-000035d9 (None) Down (None)
SIP/1_stub-000035d8 (None) Down (None)
SIP/1_stub-000035d7 (None) Down (None)
SIP/1_stub-000035d6 (None) Down (None)
SIP/1_stub-000035d5 (None) Down (None)
SIP/1_stub-000035d4 (None) Down (None)
SIP/1_stub-000035d2 (None) Down (None)
SIP/1_stub-000035c9 (None) Down (None)
SIP/iadavidenko_wof- (None) Down AppDial2((Outgoing Line))
SIP/ndmargaryan_wof- (None) Down (None)
48 active channels
0 active calls
6798 calls processed
ds-asterisk05*CLI>
{code}
3. Have errors in logs
{code}
Apr 19 14:12:59 ds-asterisk05.tcsbank.ru asterisk[11220]: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x60b00005b588 (0) at line 122 in stasis_message_create_full of stasis_message.c
Apr 19 14:12:59 ds-asterisk05.tcsbank.ru asterisk[11220]: [2017-04-19 14:12:59] ERROR[2257][C-00000518]: stasis_message.c:122 stasis_message_create_full: Excessive refcount 100000 reached on ao2 object 0x60b00005b588
Apr 19 14:12:59 ds-asterisk05.tcsbank.ru asterisk[11220]: [2017-04-19 14:12:59] ERROR[2257][C-00000518]: stasis_message.c:122 stasis_message_create_full: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x60b00005b588 (0)
{code}
> res_rtp_asterisk: Race condition when RTCP and WebRTC is used
> -------------------------------------------------------------
>
> Key: ASTERISK-26831
> URL: https://issues.asterisk.org/jira/browse/ASTERISK-26831
> Project: Asterisk
> Issue Type: Bug
> Security Level: None
> Components: Resources/res_rtp_asterisk
> Affects Versions: 13.14.0, 13.15.0
> Reporter: Badalian Vyacheslav
> Assignee: Unassigned
> Severity: Minor
> Attachments: backtrace.txt
>
>
> Race condition. With this story, I am fighting for more than a month. I tried different options but the idea of an end. There is only a heavy load (more than 200 simultaneous calls WebRTC) after 50 000-100 000 calls.
> Please help. It is difficult to disentangle this tangle. The error for a long time (many versions).
> It may have an idea why this might happen? This is because rtp_stop or because rtp_destroy?
> {code}
> =ERROR: AddressSanitizer: heap-use-after-free on address 0x6290016c326c at pc 0x7f007c288b1b bp 0x7f00404275b0 sp 0x7f0040426d58
> WRITE of size 64 at 0x6290016c326c thread T467
> #0 0x7f007c288b1a (/lib64/libasan.so.3+0x5eb1a)
> #1 0x7f0074fdea26 in memcpy /usr/include/bits/string3.h:51
> #2 0x7f0074fdea26 in ast_srtp_protect /home/pbs.vbadalyan/asterisk-13.14.0/res/res_srtp.c:438
> #3 0x7f005c6e65de in __rtp_sendto /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:2305
> #4 0x7f005c6f5e8d in rtcp_sendto /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:2330
> #5 0x7f005c6f5e8d in ast_rtcp_write_report /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:3286
> #6 0x7f005c6f6ecc in ast_rtcp_write /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:3357
> #7 0x740c64 in ast_sched_runq /home/pbs.vbadalyan/asterisk-13.14.0/main/sched.c:783
> #8 0x7f00469ad02d in do_monitor /home/pbs.vbadalyan/asterisk-13.14.0/channels/chan_sip.c:29514
> #9 0x7b217f in dummy_start /home/pbs.vbadalyan/asterisk-13.14.0/main/utils.c:1235
> #10 0x7f007a4c2dc4 in start_thread (/lib64/libpthread.so.0+0x7dc4)
> #11 0x7f00797a273c in clone (/lib64/libc.so.6+0xf773c)
> {code}
> {code}
> =ERROR: AddressSanitizer: heap-use-after-free on address 0x603002c6d820 at pc 0x7f55310f2ff3 bp 0x7f552b6f2240 sp 0x7f552b6f19e8
> READ of size 3 at 0x603002c6d820 thread T88
> #0 0x7f55310f2ff2 (/lib64/libasan.so.3+0x3cff2)
> #1 0x7f552fbae4a4 (/lib64/libjansson.so.4+0x74a4)
> #2 0x7f552fbad3f7 (/lib64/libjansson.so.4+0x63f7)
> #3 0x7f552fbad8f3 (/lib64/libjansson.so.4+0x68f3)
> #4 0x7f552fbad723 (/lib64/libjansson.so.4+0x6723)
> #5 0x7f552fbadb8c in json_vpack_ex (/lib64/libjansson.so.4+0x6b8c)
> #6 0x64fd24 in ast_json_vpack /home/pbs.vbadalyan/asterisk-13.14.0/main/json.c:823
> #7 0x64fef8 in ast_json_pack /home/pbs.vbadalyan/asterisk-13.14.0/main/json.c:814
> #8 0x7f5511e81f26 in ast_rtcp_write_report /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:3295
> #9 0x7f5511e82ea4 in ast_rtcp_write /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:3360
> #10 0x740c64 in ast_sched_runq /home/pbs.vbadalyan/asterisk-13.14.0/main/sched.c:783
> #11 0x7f54fc11a02d in do_monitor /home/pbs.vbadalyan/asterisk-13.14.0/channels/chan_sip.c:29514
> #12 0x7b217f in dummy_start /home/pbs.vbadalyan/asterisk-13.14.0/main/utils.c:1235
> #13 0x7f552f34edc4 in start_thread (/lib64/libpthread.so.0+0x7dc4)
> #14 0x7f552e62e73c in clone (/lib64/libc.so.6+0xf773c)
> {code}
> {code}
> ERROR: AddressSanitizer: heap-use-after-free on address 0x6030003739c0 at pc 0x7f30ccac6e05 bp 0x7f30bec2e2b0 sp 0x7f30bec2da58
> READ of size 19 at 0x6030003739c0 thread T738
> #0 0x7f30ccac6e04 in __asan_memcpy (/lib64/libasan.so.2+0x8ce04)
> #1 0x7f30cb530571 (/lib64/libjansson.so.4+0x5571)
> #2 0x7f30cb532887 in json_string_nocheck (/lib64/libjansson.so.4+0x7887)
> #3 0x7f30cb531907 (/lib64/libjansson.so.4+0x6907)
> #4 0x7f30cb531723 (/lib64/libjansson.so.4+0x6723)
> #5 0x7f30cb531b8c in json_vpack_ex (/lib64/libjansson.so.4+0x6b8c)
> #6 0x64c2e4 in ast_json_vpack /home/pbs.vbadalyan/asterisk-13.14.0/main/json.c:823
> #7 0x64c4b8 in ast_json_pack /home/pbs.vbadalyan/asterisk-13.14.0/main/json.c:814
> #8 0x7f30addd8d77 in ast_sockaddr_copy /home/pbs.vbadalyan/asterisk-13.14.0/include/asterisk/netsock2.h:145
> #9 0x7f30addd8d77 in ast_rtcp_read /home/pbs.vbadalyan/asterisk-13.14.0/res/res_rtp_asterisk.c:4217
> #10 0x7f30addd93cc (/usr/lib/asterisk/modules/res_rtp_asterisk.so+0x2a3cc)
> #11 0x73d1e3 in ast_sched_runq /home/pbs.vbadalyan/asterisk-13.14.0/main/sched.c:783
> #12 0x7f30972c78d5 in do_monitor /home/pbs.vbadalyan/asterisk-13.14.0/channels/chan_sip.c:29514
> #13 0x7ae78f in dummy_start /home/pbs.vbadalyan/asterisk-13.14.0/main/utils.c:1235
> #14 0x7f30cacd2dc4 in start_thread (/lib64/libpthread.so.0+0x7dc4)
> #15 0x7f30c9fb373c in clone (/lib64/libc.so.6+0xf773c)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
More information about the asterisk-bugs
mailing list