[asterisk-bugs] [JIRA] (ASTERISK-20914) Segfaults in 1.8.19.1 and 11.1.2 while on TCP SIP call

Mon Jan 14 04:38:45 CST 2013

     [ https://issues.asterisk.org/jira/browse/ASTERISK-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John McEleney updated ASTERISK-20914:
-------------------------------------

    Attachment: backtrace_1.8.19.1_2013-01-14T09_36_36+0000.txt
                capture_ending_1.8.19.1_2013-01-14T09_36_45.pcap

Over the weekend I simplfied our set-up a little by reducing us to just one IAX trunk, and using SIP for the second one. It turns out that all it takes is one IAX trunk to be in use to crash the system. The anatomy of the Segfault seems identical to the others.

I'm really very surprised at this. How can it be that on a freshly compiled Asterisk 1.8.19.1 we're unable to maintain a single IAX trunk without getting segfaults time and time again?

I occurred to me that there might be something interesting in the payload, so I captured the latest IAX call with tcpdump using a snaplen of 110, and capturing all port 4569 traffic. This seems to have captured the whole packets. I'm attaching the tail end of that capture.

There's very obvious two-way audio traffic between the two server up until a certain point. At 1358156195.623043 the last packet to come from my server is sent. I assume that that is when Asterisk segfaulted, or more likely upon receipt of the very next packet (1358156195.636835) from the other end (Gradwell, an IAX trunk provider). Both of these timestamps coincide with the timestamp of the core file.

Just to be clear, 80.248.178.80 is my Asterisk box, and 212.11.91.204 is Gradwell. There is another host 80.248.176.129 that sends a "POKE". This is another box that is configured to have an IAX trunk to my box, but that trunk is not configured at my end.

I'm attaching the backtrace as backtrace_1.8.19.1_2013-01-14T09_36_36+0000.txt.

> Segfaults in 1.8.19.1 and 11.1.2 while on TCP SIP call
> ------------------------------------------------------
>
>                 Key: ASTERISK-20914
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-20914
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Channels/chan_sip/General
>    Affects Versions: 1.8.19.1, 11.1.2
>         Environment: Two different systems. Asterisk 1.8.19.1 on CentOS 6.3 (2.6.32-279.19.1.el6.x86_64) and Asterisk 11.1.2 on CentOS 6.3 (2.6.32-279.19.1.el6.i686). Each system has a Digium TE122 card installed.
> libpri-1.4.14 and dahdi 2.6.1 installed. We're using Cisco 7961 handsets set to use SIP over TCP.
>            Reporter: John McEleney
>            Assignee: John McEleney
>         Attachments: backtrace_1.8.10.1_2013-01-10T23_27_14+0000.txt, backtrace_1.8.19.1_2013-01-14T09_36_36+0000.txt, backtrace_2013-01-10T17_49_24+0000.txt, backtrace_2013-01-10T20_39_14+0000.txt, backtrace.txt, capture_ending_1.8.19.1_2013-01-14T09_36_45.pcap
>
>
> We've been getting a lot of Segfaults on both these versions of Asterisk, although my focus at the moment is on 1.8.19.1 system. I should hopefully be able to provide backtraces from 11.1.2 later.
> This is the error in syslog:
> Jan 10 10:54:32 voip2 kernel: asterisk[4355]: segfault at 0 ip 00007fb8924c8122 sp 00007fb84440cf08 error 4 in libc-2.12.so[7fb89243f000+189000]
> In addition to the PRI card, we also use a SIP trunk, which uses UDP. Many calls will be SIP(TCP) <=> Asterisk <=> SIP(UDP).
> This problem occurs several times a day. Today it happened twice in the space of 5 minutes, but on other days it may happen between 2-10 times a day on a lightly loaded system. There tends to be no more than three concurrent calls, but the problem can occur when there is only one call. The system only ever Segfaults while someone is on a call.
> Asterisk is built with compile flags DONT_OPTIMIZE, DEBUG_THREADS and MALLOC_DEBUG.
> This is a backtrace:
> (gdb) bt
> #0  memcpy () at ../sysdeps/x86_64/memcpy.S:191
> #1  0x00000000004d333d in ast_frdup (f=0x7fb81400a428) at frame.c:537
> #2  0x000000000046cad6 in __ast_queue_frame (chan=0x7fb8780b0ba0, fin=0x7fb81400a428, head=0, after=0x0) at channel.c:1483
> #3  0x000000000046d125 in ast_queue_frame (chan=0x7fb8780b0ba0, fin=0x7fb81400a428) at channel.c:1555
> #4  0x00007fb883b72335 in local_queue_frame (p=0x7fb8780a9ef0, isoutbound=0, f=0x7fb81400a428, us=0x7fb8780aad10, us_locked=1) at chan_local.c:447
> #5  0x00007fb883b73192 in local_write (ast=0x7fb8780aad10, f=0x7fb81400a428) at chan_local.c:660
> #6  0x0000000000477e0d in ast_write (chan=0x7fb8780aad10, fr=0x7fb81400a428) at channel.c:5109
> #7  0x00000000004802d3 in ast_generic_bridge (c0=0x7fb81c008990, c1=0x7fb8780aad10, config=0x7fb84440f920, fo=0x7fb84440dcd8, rc=0x7fb84440dcd0)
>     at channel.c:7260
> #8  0x00000000004822a9 in ast_channel_bridge (c0=0x7fb81c008990, c1=0x7fb8780aad10, config=0x7fb84440f920, fo=0x7fb84440dcd8, rc=0x7fb84440dcd0)
>     at channel.c:7618
> #9  0x00000000004c0b23 in ast_bridge_call (chan=0x7fb81c008990, peer=0x7fb8780aad10, config=0x7fb84440f920) at features.c:4102
> #10 0x00007fb852631dc6 in try_calling (qe=0x7fb84440fff0, options=0x7fb84440ff85 "", announceoverride=0x7fb84440ff87 "", url=0x7fb84440ff86 "", 
>     tries=0x7fb8444111e4, noption=0x7fb8444111e0, agi=0x7fb84440ff8c "", macro=0x7fb84440ff8d "", gosub=0x7fb84440ff8e "", ringing=0) at app_queue.c:5243
> #11 0x00007fb852635a9c in queue_exec (chan=0x7fb81c008990, data=0x7fb844413460 "123,t,,,240,,,,,") at app_queue.c:6194
> #12 0x0000000000508c8e in pbx_exec (c=0x7fb81c008990, app=0x2d50528, data=0x7fb844413460 "123,t,,,240,,,,,") at pbx.c:1447
> #13 0x0000000000512ee2 in pbx_extension_helper (c=0x7fb81c008990, con=0x0, context=0x7fb81c008f00 "ext-queues", exten=0x7fb81c008f50 "123", priority=23, 
>     label=0x0, callerid=0x7fb878069078 "01638780352", action=E_SPAWN, found=0x7fb844415b58, combined_find_spawn=1) at pbx.c:4267
> #14 0x0000000000514c01 in ast_spawn_extension (c=0x7fb81c008990, context=0x7fb81c008f00 "ext-queues", exten=0x7fb81c008f50 "123", priority=23, 
>     callerid=0x7fb878069078 "01638780352", found=0x7fb844415b58, combined_find_spawn=1) at pbx.c:4907
> #15 0x00000000005156b7 in __ast_pbx_run (c=0x7fb81c008990, args=0x0) at pbx.c:5010
> #16 0x0000000000517412 in pbx_thread (data=0x7fb81c008990) at pbx.c:5351
> #17 0x000000000056ef88 in dummy_start (data=0x7fb81c004728) at utils.c:1012
> #18 0x00007fb891839851 in start_thread (arg=0x7fb844416700) at pthread_create.c:301
> #19 0x00007fb89252711d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
> Any help with this would be greatly appreciated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira