[asterisk-bugs] [JIRA] (ASTERISK-29116) Asterisk hang in pbx.c on thread exit

Kevin Harwell (JIRA) noreply at issues.asterisk.org
Wed Oct 28 09:55:15 CDT 2020


    [ https://issues.asterisk.org/jira/browse/ASTERISK-29116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=252590#comment-252590 ] 

Kevin Harwell edited comment on ASTERISK-29116 at 10/28/20 9:53 AM:
--------------------------------------------------------------------

The attachment just contains the core file. We cannot make use of that locally without all the backing symbol files. Even then it can be a tricky endeavor to load correctly.

The simplest thing is to use the core file from the system it was produced on. The script will automatically extract information (e.g. backtrace) from the core that could be helpful.

Edit: Or what [~seanbright] said :-)


was (Author: kharwell):
The attachment just contains the core file. We cannot make use of that locally without all the backing symbol files. Even then it can be a tricky endeavor to load correctly.

The simplest thing is to use the core file from the system it was produced on. The script will automatically extract information (e.g. backtrace) from the core that could be helpful.

Edit: Or what @sean bright said :-)

> Asterisk hang in pbx.c on thread exit
> -------------------------------------
>
>                 Key: ASTERISK-29116
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-29116
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: PBX/General
>    Affects Versions: 17.5.1
>            Reporter: Dennis Haney
>            Assignee: Dennis Haney
>            Severity: Minor
>         Attachments: core-asterisk-running-2020-10-26T16-04-34+0100.gz
>
>
> I have hundreds of threads stuck with this stack trace, which eventually causes the server to run out of resources and is unable to make further calls.
> I dont see any other threads with dl_open in the stack that isnt this, nor any other threads with pbx_thread that isnt stuck like this.
> FYI, we are stuck on chan_sip if that makes a difference.
> {noformat}
> Thread 686 (Thread 0x7f863d3d6700 (LWP 14068)):
> #0  0x00007f867b73129c in __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
> #1  0x00007f867b72a7d1 in __GI___pthread_mutex_lock (mutex=0x7f867c080968 <_rtld_global+2312>) at ../nptl/pthread_mutex_lock.c:115
>         id = 14068
>         type = <optimized out>
>         __PRETTY_FUNCTION__ = "__pthread_mutex_lock"
>         id = <optimized out>
> #2  0x00007f867c06ab56 in _dl_open (file=0x7f867b735a0f "libgcc_s.so.1", mode=-2147483646, caller_dlopen=0x7f867b732c7b <pthread_cancel_init+43>, nsid=-2, argc=1, argv=0x7ffda
> dfe0b08, env=0x563a65927300) at dl-open.c:548
>         args = {file = 0x0, mode = -1, caller_dlopen = 0x0, map = 0x563a64441e93 <ast_unref_namedgroups+35>, nsid = 140215383486384, argc = 1682268464, argv = 0x7f86701fefb0,
> env = 0x7f86701fefb0}
>         exception = {objname = 0x563a646873e0 <change_pending> "376036", errstring = 0x563a645d5c30 "devicestate.c", message_buffer = 0x7f86701fef98 "340263Cd:V"}
>         errcode = <optimized out>
>         __PRETTY_FUNCTION__ = "_dl_open"
> #3  0x00007f867b33906d in do_dlopen (ptr=ptr at entry=0x7f863d3d5d40) at dl-libc.c:96
>         args = 0x7f863d3d5d40
> #4  0x00007f867b339b2f in __GI__dl_catch_exception (exception=exception at entry=0x7f863d3d5cc0, operate=operate at entry=0x7f867b339030 <do_dlopen>, args=args at entry=0x7f863d3d5d40)
>  at dl-error-skeleton.c:196
>         errcode = 22074
>         c = {exception = 0x7f863d3d5cc0, errcode = 0x7f863d3d5bcc, env = {{__jmpbuf = {0, -5657367600969769489, 140214529776920, 140214527728383, 140214529779456, 140215116599
> 168, -5657367600885883409, -5657521440193339921}, __mask_was_saved = 0, __saved_mask = {__val = {140215383488960, 0, 94808791407956, 140215383486384, 94808791900464, 140215383
> 488960, 140215383486384, 0, 140215383488960, 0, 0, 140214529776828, 94808790813846, 140214529776832, 0, 0}}}}}
>         old = <optimized out>
> #5  0x00007f867b339bbf in __GI__dl_catch_error (objname=objname at entry=0x7f863d3d5d18, errstring=errstring at entry=0x7f863d3d5d20, mallocedp=mallocedp at entry=0x7f863d3d5d17, opera
> te=operate at entry=0x7f867b339030 <do_dlopen>, args=args at entry=0x7f863d3d5d40) at dl-error-skeleton.c:215
>         exception = {objname = 0x563a645f2200 <__PRETTY_FUNCTION__.17003+32> "he", errstring = 0xd2646d2b5cddd100 <error: Cannot access memory at address 0xd2646d2b5cddd100>,
> message_buffer = 0x20 <error: Cannot access memory at address 0x20>}
>         errorcode = <optimized out>
> #6  0x00007f867b339147 in dlerror_run (operate=operate at entry=0x7f867b339030 <do_dlopen>, args=args at entry=0x7f863d3d5d40) at dl-libc.c:46
>         objname = 0x563a645e0713 "pbx.c"
>         last_errstring = 0x0
>         malloced = false
>         result = <optimized out>
> #7  0x00007f867b3391d6 in __GI___libc_dlopen_mode (name=name at entry=0x7f867b735a0f "libgcc_s.so.1", mode=mode at entry=-2147483646) at dl-libc.c:195
>         args = {name = 0x7f867b735a0f "libgcc_s.so.1", mode = -2147483646, caller_dlopen = 0x7f867b732c7b <pthread_cancel_init+43>, map = 0x7f8674000020}
> #8  0x00007f867b732c7b in pthread_cancel_init () at ../sysdeps/nptl/unwind-forcedunwind.c:53
>         resume = <optimized out>
>         personality = <optimized out>
>         forcedunwind = <optimized out>
>         getcfa = <optimized out>
>         handle = <optimized out>
> #9  0x00007f867b732e94 in _Unwind_ForcedUnwind (exc=0x7f863d3d6d70, stop=0x7f867b730f30 <unwind_stop>, stop_argument=0x7f863d3d5e30) at ../sysdeps/nptl/unwind-forcedunwind.c:127
>         forcedunwind = <optimized out>
> #10 0x00007f867b7310a0 in __GI___pthread_unwind (buf=<optimized out>) at unwind.c:121
>         ibuf = <optimized out>
>         self = <optimized out>
> #11 0x00007f867b7291f5 in __do_cancel () at pthreadP.h:305
> #12 0x00007f867b7291f5 in __pthread_exit (value=value at entry=0x0) at pthread_exit.c:28
> #13 0x0000563a644c59f1 in pbx_thread (data=data at entry=0x7f86701fefb0) at pbx.c:4698
>         c = 0x7f86701fefb0
> #14 0x0000563a64542fec in dummy_start (data=<optimized out>) at utils.c:1249
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {140215317300000, 5652163074533400047, 140214527728382, 140214527728383, 140214529779456, 140215116599168, -5657367601009615377, -2160517297729790481}, __mask_was_saved = 0}}, __pad = {0x7f863d3d5ed0, 0x0, 0x7f863d3d6700, 0x7c000}}
>         __cancel_arg = 0x7f863d3d6700
>         __not_first_call = <optimized out>
>         ret = <optimized out>
>         a = {start_routine = 0x563a644c5980 <pbx_thread>, data = 0x7f86701fefb0, name = <optimized out>}
>         __PRETTY_FUNCTION__ = "dummy_start"
> #15 0x00007f867b727fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
>         ret = <optimized out>
>         pd = <optimized out>
>         now = <optimized out>
>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140214529779456, 5652163074533400047, 140214527728382, 140214527728383, 140214529779456, 140215116599168, -5657367600982352401, -5657520887340556817}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
>         not_first_call = <optimized out>
> #16 0x00007f867b2fe4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)



More information about the asterisk-bugs mailing list