[asterisk-bugs] [JIRA] (ASTERISK-26186) res_pjsip_multihomed: Deadlock when finding message header
Mateusz Kowalski (JIRA)
noreply at issues.asterisk.org
Mon Jul 11 09:41:57 CDT 2016
[ https://issues.asterisk.org/jira/browse/ASTERISK-26186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mateusz Kowalski updated ASTERISK-26186:
----------------------------------------
Description:
Hello,
I encountered a deadlock in PJSIP module. Two backtraces available as
* http://pastebin.com/raw/kZtc5df3 (Thread 43)
* http://pastebin.com/raw/93H9qe4x (Thread 42)
Output from {{core show locks}} is empty:
{code}
tone-0058-re-1*CLI> core show locks
=======================================================================
=== 13.9.1
=== Currently Held Locks
=======================================================================
===
=== <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name> <lock addr> (times locked)
===
=======================================================================
{code}
Any packets arriving to the machine on port 5060 are left without any response. They are not visible in Asterisk console even with {{pjsip set logger on}}
Output from {{netstat}} shows big RX queue for the process:
{code}
udp 214912 0 0.0.0.0:5060 0.0.0.0:* 46787/asterisk
{code}
If you check {{netstat}} per port, it happens also on RTP port (I'm using 10000-20000):
{code}
udp 0 0 0.0.0.0:2727 0.0.0.0:* 46787/asterisk
udp 896 0 0.0.0.0:35859 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:37171 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:4520 0.0.0.0:* 46787/asterisk
udp 214912 0 0.0.0.0:5060 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:10520 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:10521 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:11356 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:11357 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12048 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12049 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12062 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12063 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:13890 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:13891 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14608 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14609 0.0.0.0:* 46787/asterisk
udp 213248 0 0.0.0.0:14678 0.0.0.0:* 46787/asterisk
udp 34048 0 0.0.0.0:14679 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14752 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14753 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:15106 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:15107 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18100 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18101 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18392 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18393 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18918 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18919 0.0.0.0:* 46787/asterisk
{code}
Initially I reported issue on #asterisk but later on got redirected to open a ticket in here.
Snaps of my configuration in here:
{{asterisk.conf}}
{code}
[directories](!)
astetcdir => /etc/asterisk
astmoddir => /usr/lib/asterisk/modules
astvarlibdir => /var/lib/asterisk
astdbdir => /var/lib/asterisk
astkeydir => /var/lib/asterisk
astdatadir => /var/lib/asterisk
astagidir => /var/lib/asterisk/agi-bin
astspooldir => /var/spool/asterisk
astrundir => /var/run/asterisk
astlogdir => /var/log/asterisk
astsbindir => /usr/sbin
[options]
;verbose = 5
;debug = 5
;alwaysfork = yes ; Same as -F at startup.
;nofork = yes ; Same as -f at startup.
;quiet = yes ; Same as -q at startup.
;timestamp = yes ; Same as -T at startup.
;execincludes = yes ; Support #exec in config files.
;console = yes ; Run as console (same as -c at startup).
;highpriority = yes ; Run realtime priority (same as -p at
; startup).
;initcrypto = yes ; Initialize crypto keys (same as -i at
; startup).
;nocolor = yes ; Disable console colors.
;dontwarn = yes ; Disable some warnings.
;dumpcore = yes ; Dump core on crash (same as -g at startup).
;languageprefix = yes ; Use the new sound prefix path syntax.
;systemname = my_system_name ; Prefix uniqueid with a system name for
; Global uniqueness issues.
;autosystemname = yes ; Automatically set systemname to hostname,
; uses 'localhost' on failure, or systemname if
; set.
;mindtmfduration = 80 ; Set minimum DTMF duration in ms (default 80 ms)
; If we get shorter DTMF messages, these will be
; changed to the minimum duration
;maxcalls = 10 ; Maximum amount of calls allowed.
;maxload = 0.9 ; Asterisk stops accepting new calls if the
; load average exceed this limit.
;maxfiles = 1000 ; Maximum amount of openfiles.
;minmemfree = 1 ; In MBs, Asterisk stops accepting new calls if
; the amount of free memory falls below this
; watermark.
;cache_record_files = yes ; Cache recorded sound files to another
; directory during recording.
;record_cache_dir = /tmp ; Specify cache directory (used in conjunction
; with cache_record_files).
;transmit_silence = yes ; Transmit silence while a channel is in a
; waiting state, a recording only state, or
; when DTMF is being generated. Note that the
; silence internally is generated in raw signed
; linear format. This means that it must be
; transcoded into the native format of the
; channel before it can be sent to the device.
; It is for this reason that this is optional,
; as it may result in requiring a temporary
; codec translation path for a channel that may
; not otherwise require one.
;transcode_via_sln = yes ; Build transcode paths via SLINEAR, instead of
; directly.
;runuser = asterisk ; The user to run as.
;rungroup = asterisk ; The group to run as.
;lightbackground = yes ; If your terminal is set for a light-colored
; background.
;forceblackbackground = yes ; Force the background of the terminal to be
; black, in order for terminal colors to show
; up properly.
;defaultlanguage = en ; Default language
documentation_language = en_US ; Set the language you want documentation
; displayed in. Value is in the same format as
; locale names.
;hideconnect = yes ; Hide messages displayed when a remote console
; connects and disconnects.
;lockconfdir = no ; Protect the directory containing the
; configuration files (/etc/asterisk) with a
; lock.
;stdexten = gosub ; How to invoke the extensions.conf stdexten.
; macro - Invoke the stdexten using a macro as
; done by legacy Asterisk versions.
; gosub - Invoke the stdexten using a gosub as
; documented in extensions.conf.sample.
; Default gosub.
;live_dangerously = no ; Enable the execution of 'dangerous' dialplan
; functions from external sources (AMI,
; etc.) These functions (such as SHELL) are
; considered dangerous because they can allow
; privilege escalation.
; Default no
; Changing the following lines may compromise your security.
;[files]
;astctlpermissions = 0660
;astctlowner = root
;astctlgroup = apache
;astctl = asterisk.ctl
{code}
{{extensions.conf}}
{code}
[professional]
switch => Realtime/professional at extensions
[private]
switch => Realtime/private at extensions
[sigos]
switch => Realtime/sigos at extensions
{code}
{{hep.conf}}
{code}
[general]
enabled = yes
capture_address = 188.184.65.170:9060
;capture_password = foo
capture_id = 1234
{code}
{{modules.conf}}
{code}
[modules]
autoload=yes
preload => res_odbc.so
preload => res_config_odbc.so
{code}
{{pjsip.conf}}
{code}
[transport-udp]
type=transport
protocol=udp
bind=0.0.0.0
{code}
{{sorcery.conf}}
{code}
[res_pjsip]
endpoint=realtime,ps_endpoints
;endpoint=config,pjsip.conf,criteria=type=endpoint
;auth=realtime,ps_auths
aor=realtime,ps_aors
;aor=config,pjsip.conf,criteria=type=aor
;domain_alias=realtime,ps_domain_aliases
;contact=realtime,ps_contacts
[res_pjsip_endpoint_identifier_ip]
identify=realtime,ps_endpoint_id_ips
;identify=config,pjsip.conf,criteria=type=identify
{code}
Thanks for any help,
Mateusz
was:
Hello,
I encountered a deadlock in PJSIP module. Two backtraces available as
* http://pastebin.com/raw/kZtc5df3 (Thread 43)
* http://pastebin.com/raw/93H9qe4x (Thread 42)
Output from {{core show locks}} is empty:
{code}
tone-0058-re-1*CLI> core show locks
=======================================================================
=== 13.9.1
=== Currently Held Locks
=======================================================================
===
=== <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name> <lock addr> (times locked)
===
=======================================================================
{code}
Any packets arriving to the machine on port 5060 are left without any response. They are not visible in Asterisk console even with {{pjsip set logger on}}
Output from {{netstat}} shows big RX queue for the process:
{code}
udp 214912 0 0.0.0.0:5060 0.0.0.0:* 46787/asterisk
{code}
If you check {{netstat}} per port, it happens also on RTP port (I'm using 10000-20000):
{code}
udp 0 0 0.0.0.0:2727 0.0.0.0:* 46787/asterisk
udp 896 0 0.0.0.0:35859 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:37171 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:4520 0.0.0.0:* 46787/asterisk
udp 214912 0 0.0.0.0:5060 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:10520 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:10521 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:11356 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:11357 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12048 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12049 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12062 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:12063 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:13890 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:13891 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14608 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14609 0.0.0.0:* 46787/asterisk
udp 213248 0 0.0.0.0:14678 0.0.0.0:* 46787/asterisk
udp 34048 0 0.0.0.0:14679 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14752 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:14753 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:15106 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:15107 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18100 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18101 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18392 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18393 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18918 0.0.0.0:* 46787/asterisk
udp 0 0 0.0.0.0:18919 0.0.0.0:* 46787/asterisk
{code}
Initially I reported issue on #asterisk but later on got redirected to open a ticket in here.
Thanks for any help,
Mateusz
> res_pjsip_multihomed: Deadlock when finding message header
> ----------------------------------------------------------
>
> Key: ASTERISK-26186
> URL: https://issues.asterisk.org/jira/browse/ASTERISK-26186
> Project: Asterisk
> Issue Type: Bug
> Security Level: None
> Components: pjproject/pjsip
> Affects Versions: 13.9.1
> Environment: CentOS 7.2 (3.10.0-327.22.2.el7.x86_64)
> Asterisk 13.9.1
> PJSIP 2.5.1
> Reporter: Mateusz Kowalski
> Assignee: Unassigned
> Attachments: trace1.txt, trace2.txt
>
>
> Hello,
> I encountered a deadlock in PJSIP module. Two backtraces available as
> * http://pastebin.com/raw/kZtc5df3 (Thread 43)
> * http://pastebin.com/raw/93H9qe4x (Thread 42)
> Output from {{core show locks}} is empty:
> {code}
> tone-0058-re-1*CLI> core show locks
> =======================================================================
> === 13.9.1
> === Currently Held Locks
> =======================================================================
> ===
> === <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name> <lock addr> (times locked)
> ===
> =======================================================================
> {code}
> Any packets arriving to the machine on port 5060 are left without any response. They are not visible in Asterisk console even with {{pjsip set logger on}}
> Output from {{netstat}} shows big RX queue for the process:
> {code}
> udp 214912 0 0.0.0.0:5060 0.0.0.0:* 46787/asterisk
> {code}
> If you check {{netstat}} per port, it happens also on RTP port (I'm using 10000-20000):
> {code}
> udp 0 0 0.0.0.0:2727 0.0.0.0:* 46787/asterisk
> udp 896 0 0.0.0.0:35859 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:37171 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:4520 0.0.0.0:* 46787/asterisk
> udp 214912 0 0.0.0.0:5060 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:10520 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:10521 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:11356 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:11357 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:12048 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:12049 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:12062 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:12063 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:13890 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:13891 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:14608 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:14609 0.0.0.0:* 46787/asterisk
> udp 213248 0 0.0.0.0:14678 0.0.0.0:* 46787/asterisk
> udp 34048 0 0.0.0.0:14679 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:14752 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:14753 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:15106 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:15107 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:18100 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:18101 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:18392 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:18393 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:18918 0.0.0.0:* 46787/asterisk
> udp 0 0 0.0.0.0:18919 0.0.0.0:* 46787/asterisk
> {code}
> Initially I reported issue on #asterisk but later on got redirected to open a ticket in here.
> Snaps of my configuration in here:
> {{asterisk.conf}}
> {code}
> [directories](!)
> astetcdir => /etc/asterisk
> astmoddir => /usr/lib/asterisk/modules
> astvarlibdir => /var/lib/asterisk
> astdbdir => /var/lib/asterisk
> astkeydir => /var/lib/asterisk
> astdatadir => /var/lib/asterisk
> astagidir => /var/lib/asterisk/agi-bin
> astspooldir => /var/spool/asterisk
> astrundir => /var/run/asterisk
> astlogdir => /var/log/asterisk
> astsbindir => /usr/sbin
> [options]
> ;verbose = 5
> ;debug = 5
> ;alwaysfork = yes ; Same as -F at startup.
> ;nofork = yes ; Same as -f at startup.
> ;quiet = yes ; Same as -q at startup.
> ;timestamp = yes ; Same as -T at startup.
> ;execincludes = yes ; Support #exec in config files.
> ;console = yes ; Run as console (same as -c at startup).
> ;highpriority = yes ; Run realtime priority (same as -p at
> ; startup).
> ;initcrypto = yes ; Initialize crypto keys (same as -i at
> ; startup).
> ;nocolor = yes ; Disable console colors.
> ;dontwarn = yes ; Disable some warnings.
> ;dumpcore = yes ; Dump core on crash (same as -g at startup).
> ;languageprefix = yes ; Use the new sound prefix path syntax.
> ;systemname = my_system_name ; Prefix uniqueid with a system name for
> ; Global uniqueness issues.
> ;autosystemname = yes ; Automatically set systemname to hostname,
> ; uses 'localhost' on failure, or systemname if
> ; set.
> ;mindtmfduration = 80 ; Set minimum DTMF duration in ms (default 80 ms)
> ; If we get shorter DTMF messages, these will be
> ; changed to the minimum duration
> ;maxcalls = 10 ; Maximum amount of calls allowed.
> ;maxload = 0.9 ; Asterisk stops accepting new calls if the
> ; load average exceed this limit.
> ;maxfiles = 1000 ; Maximum amount of openfiles.
> ;minmemfree = 1 ; In MBs, Asterisk stops accepting new calls if
> ; the amount of free memory falls below this
> ; watermark.
> ;cache_record_files = yes ; Cache recorded sound files to another
> ; directory during recording.
> ;record_cache_dir = /tmp ; Specify cache directory (used in conjunction
> ; with cache_record_files).
> ;transmit_silence = yes ; Transmit silence while a channel is in a
> ; waiting state, a recording only state, or
> ; when DTMF is being generated. Note that the
> ; silence internally is generated in raw signed
> ; linear format. This means that it must be
> ; transcoded into the native format of the
> ; channel before it can be sent to the device.
> ; It is for this reason that this is optional,
> ; as it may result in requiring a temporary
> ; codec translation path for a channel that may
> ; not otherwise require one.
> ;transcode_via_sln = yes ; Build transcode paths via SLINEAR, instead of
> ; directly.
> ;runuser = asterisk ; The user to run as.
> ;rungroup = asterisk ; The group to run as.
> ;lightbackground = yes ; If your terminal is set for a light-colored
> ; background.
> ;forceblackbackground = yes ; Force the background of the terminal to be
> ; black, in order for terminal colors to show
> ; up properly.
> ;defaultlanguage = en ; Default language
> documentation_language = en_US ; Set the language you want documentation
> ; displayed in. Value is in the same format as
> ; locale names.
> ;hideconnect = yes ; Hide messages displayed when a remote console
> ; connects and disconnects.
> ;lockconfdir = no ; Protect the directory containing the
> ; configuration files (/etc/asterisk) with a
> ; lock.
> ;stdexten = gosub ; How to invoke the extensions.conf stdexten.
> ; macro - Invoke the stdexten using a macro as
> ; done by legacy Asterisk versions.
> ; gosub - Invoke the stdexten using a gosub as
> ; documented in extensions.conf.sample.
> ; Default gosub.
> ;live_dangerously = no ; Enable the execution of 'dangerous' dialplan
> ; functions from external sources (AMI,
> ; etc.) These functions (such as SHELL) are
> ; considered dangerous because they can allow
> ; privilege escalation.
> ; Default no
> ; Changing the following lines may compromise your security.
> ;[files]
> ;astctlpermissions = 0660
> ;astctlowner = root
> ;astctlgroup = apache
> ;astctl = asterisk.ctl
> {code}
> {{extensions.conf}}
> {code}
> [professional]
> switch => Realtime/professional at extensions
> [private]
> switch => Realtime/private at extensions
> [sigos]
> switch => Realtime/sigos at extensions
> {code}
> {{hep.conf}}
> {code}
> [general]
> enabled = yes
> capture_address = 188.184.65.170:9060
> ;capture_password = foo
> capture_id = 1234
> {code}
> {{modules.conf}}
> {code}
> [modules]
> autoload=yes
> preload => res_odbc.so
> preload => res_config_odbc.so
> {code}
> {{pjsip.conf}}
> {code}
> [transport-udp]
> type=transport
> protocol=udp
> bind=0.0.0.0
> {code}
> {{sorcery.conf}}
> {code}
> [res_pjsip]
> endpoint=realtime,ps_endpoints
> ;endpoint=config,pjsip.conf,criteria=type=endpoint
> ;auth=realtime,ps_auths
> aor=realtime,ps_aors
> ;aor=config,pjsip.conf,criteria=type=aor
> ;domain_alias=realtime,ps_domain_aliases
> ;contact=realtime,ps_contacts
> [res_pjsip_endpoint_identifier_ip]
> identify=realtime,ps_endpoint_id_ips
> ;identify=config,pjsip.conf,criteria=type=identify
> {code}
> Thanks for any help,
> Mateusz
--
This message was sent by Atlassian JIRA
(v6.2#6252)
More information about the asterisk-bugs
mailing list