[asterisk-bugs] [JIRA] (ASTERISK-30445) stasis: Failure to allocate memory causes crashes

Tue Feb 28 05:01:03 CST 2023

     [ https://issues.asterisk.org/jira/browse/ASTERISK-30445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quality Unit updated ASTERISK-30445:
------------------------------------

    Description: 
This started to happen after upgrade to `18.16.0` from `16.29.0`. Asterisk starting to spam logs with similar messages like:

{code:none}
[2023-02-24 09:30:57.001] ERROR[11361] stasis.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x7f7ec4031d10 (0)
[2023-02-24 09:30:57.006] ERROR[11361] : Got 14 backtrace records
# 0: /usr/sbin/asterisk() [0x459ae6]
# 1: /usr/sbin/asterisk(__ao2_ref+0x79) [0x459fc9]
# 2: /usr/sbin/asterisk(stasis_forward_all+0x83) [0x555283]
# 3: /usr/lib64/asterisk/modules/res_stasis.so(+0x107e1) [0x7f7f62a7e7e1]
# 4: /usr/lib64/asterisk/modules/res_stasis.so(+0x14e2a) [0x7f7f62a82e2a]
# 5: /usr/lib64/asterisk/modules/res_stasis.so(+0x12966) [0x7f7f62a80966]
# 6: /usr/lib64/asterisk/modules/res_stasis.so(+0x1554e) [0x7f7f62a8354e]
# 7: /usr/lib64/asterisk/modules/res_stasis.so(stasis_app_exec+0x569) [0x7f7f62a79789]
# 8: /usr/lib64/asterisk/modules/app_stasis.so(+0xbd7) [0x7f7f4d6f4bd7]
# 9: /usr/sbin/asterisk(pbx_exec+0xb9) [0x50db89]
#10: /usr/lib64/asterisk/modules/res_ari_channels.so(+0xa528) [0x7f7f48594528]
#11: /usr/sbin/asterisk() [0x57e80d]
#12: /lib64/libpthread.so.0(+0x7ea5) [0x7f7fb932fea5]
#13: /lib64/libc.so.6(clone+0x6d) [0x7f7fb88ff98d]
{code}

After around ~3 minutes of these (Asterisk during that time is already unresponsive), we have final errors before service crash:

{code:none}
[2023-02-24 09:32:48.153] ERROR[11361] astmm.c: Memory Allocation Failure in function topic_add_subscription at line 1214 of stasis.c
{code}

And, from `/var/log/messages`:

{code:none}
Feb 24 09:32:48 localhost kernel: asterisk[11361]: segfault at 1 ip 0000000000552827 sp 00007f7ed3e0a6b0 error 4 in asterisk[400000+2a2000]
{code}

It happens randomly, once, twice a day, sometime doesn't happen at all.
Though, it's always happening when Asterisk is processing calls, so it's not happening during night (when asterisk is almost idle)

  was:
This started to happen after upgrade to `18.16.0` from `16.29.0`. Asterisk starting to spam logs with similar messages like:

{code:none}
[2023-02-24 09:30:57.001] ERROR[11361] stasis.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x7f7ec4031d10 (0)
[2023-02-24 09:30:57.006] ERROR[11361] : Got 14 backtrace records
# 0: /usr/sbin/asterisk() [0x459ae6]
# 1: /usr/sbin/asterisk(__ao2_ref+0x79) [0x459fc9]
# 2: /usr/sbin/asterisk(stasis_forward_all+0x83) [0x555283]
# 3: /usr/lib64/asterisk/modules/res_stasis.so(+0x107e1) [0x7f7f62a7e7e1]
# 4: /usr/lib64/asterisk/modules/res_stasis.so(+0x14e2a) [0x7f7f62a82e2a]
# 5: /usr/lib64/asterisk/modules/res_stasis.so(+0x12966) [0x7f7f62a80966]
# 6: /usr/lib64/asterisk/modules/res_stasis.so(+0x1554e) [0x7f7f62a8354e]
# 7: /usr/lib64/asterisk/modules/res_stasis.so(stasis_app_exec+0x569) [0x7f7f62a79789]
# 8: /usr/lib64/asterisk/modules/app_stasis.so(+0xbd7) [0x7f7f4d6f4bd7]
# 9: /usr/sbin/asterisk(pbx_exec+0xb9) [0x50db89]
#10: /usr/lib64/asterisk/modules/res_ari_channels.so(+0xa528) [0x7f7f48594528]
#11: /usr/sbin/asterisk() [0x57e80d]
#12: /lib64/libpthread.so.0(+0x7ea5) [0x7f7fb932fea5]
#13: /lib64/libc.so.6(clone+0x6d) [0x7f7fb88ff98d]
{code}

After around ~3 minutes of these (Asterisk during that time is already unresponsive), we have final errors before service crush:

{code:none}
[2023-02-24 09:32:48.153] ERROR[11361] astmm.c: Memory Allocation Failure in function topic_add_subscription at line 1214 of stasis.c
{code}

And, from `/var/log/messages`:

{code:none}
Feb 24 09:32:48 localhost kernel: asterisk[11361]: segfault at 1 ip 0000000000552827 sp 00007f7ed3e0a6b0 error 4 in asterisk[400000+2a2000]
{code}

It happens randomly, once, twice a day, sometime doesn't happen at all.
Though, it's always happening when Asterisk is processing calls, so it's not happening during night (when asterisk is almost idle)

> stasis: Failure to allocate memory causes crashes
> -------------------------------------------------
>
>                 Key: ASTERISK-30445
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-30445
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Resources/res_stasis
>    Affects Versions: 18.16.0
>         Environment: CentOS 7, kernel 3.10.0-1160.83.1.el7.x86_64.
>            Reporter: Quality Unit
>            Assignee: Quality Unit
>            Severity: Major
>         Attachments: core_asterisk.zip
>
>
> This started to happen after upgrade to `18.16.0` from `16.29.0`. Asterisk starting to spam logs with similar messages like:
> {code:none}
> [2023-02-24 09:30:57.001] ERROR[11361] stasis.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x7f7ec4031d10 (0)
> [2023-02-24 09:30:57.006] ERROR[11361] : Got 14 backtrace records
> # 0: /usr/sbin/asterisk() [0x459ae6]
> # 1: /usr/sbin/asterisk(__ao2_ref+0x79) [0x459fc9]
> # 2: /usr/sbin/asterisk(stasis_forward_all+0x83) [0x555283]
> # 3: /usr/lib64/asterisk/modules/res_stasis.so(+0x107e1) [0x7f7f62a7e7e1]
> # 4: /usr/lib64/asterisk/modules/res_stasis.so(+0x14e2a) [0x7f7f62a82e2a]
> # 5: /usr/lib64/asterisk/modules/res_stasis.so(+0x12966) [0x7f7f62a80966]
> # 6: /usr/lib64/asterisk/modules/res_stasis.so(+0x1554e) [0x7f7f62a8354e]
> # 7: /usr/lib64/asterisk/modules/res_stasis.so(stasis_app_exec+0x569) [0x7f7f62a79789]
> # 8: /usr/lib64/asterisk/modules/app_stasis.so(+0xbd7) [0x7f7f4d6f4bd7]
> # 9: /usr/sbin/asterisk(pbx_exec+0xb9) [0x50db89]
> #10: /usr/lib64/asterisk/modules/res_ari_channels.so(+0xa528) [0x7f7f48594528]
> #11: /usr/sbin/asterisk() [0x57e80d]
> #12: /lib64/libpthread.so.0(+0x7ea5) [0x7f7fb932fea5]
> #13: /lib64/libc.so.6(clone+0x6d) [0x7f7fb88ff98d]
> {code}
> After around ~3 minutes of these (Asterisk during that time is already unresponsive), we have final errors before service crash:
> {code:none}
> [2023-02-24 09:32:48.153] ERROR[11361] astmm.c: Memory Allocation Failure in function topic_add_subscription at line 1214 of stasis.c
> {code}
> And, from `/var/log/messages`:
> {code:none}
> Feb 24 09:32:48 localhost kernel: asterisk[11361]: segfault at 1 ip 0000000000552827 sp 00007f7ed3e0a6b0 error 4 in asterisk[400000+2a2000]
> {code}
> It happens randomly, once, twice a day, sometime doesn't happen at all.
> Though, it's always happening when Asterisk is processing calls, so it's not happening during night (when asterisk is almost idle)

--
This message was sent by Atlassian JIRA
(v6.2#6252)