[asterisk-bugs] [Asterisk 0017185]: [patch] [regression] Using Local channels with queues causes deadlocks
Asterisk Bug Tracker
noreply at bugs.digium.com
Wed Apr 28 16:26:32 CDT 2010
A NOTE has been added to this issue.
======================================================================
https://issues.asterisk.org/view.php?id=17185
======================================================================
Reported By: schmoozecom
Assigned To: dvossel
======================================================================
Project: Asterisk
Issue ID: 17185
Category: Channels/chan_local
Reproducibility: sometimes
Severity: block
Priority: normal
Status: closed
Target Version: 1.4.31
Asterisk Version: 1.4.30
JIRA: SWP-1295
Regression: No
Reviewboard Link:
SVN Branch (only for SVN checkouts, not tarball releases): N/A
SVN Revision (number only!):
Request Review:
Resolution: fixed
Fixed in Version:
======================================================================
Date Submitted: 2010-04-14 16:23 CDT
Last Modified: 2010-04-28 16:26 CDT
======================================================================
Summary: [patch] [regression] Using Local channels with
queues causes deadlocks
Description:
Since upgrading to asterisk 1.4.30 we are seeing calls that get answered in
the queue still show as calls waiting in the queue. We have seen this on 2
systems now and downgrading to 1.4.29 solves the issue. When looking at
"Show Queue" it will show a dahdi channel as still waiting in the queue
even though the call has been answered and hung up.
Below is a print out of "core show locks" when asterisk is in this state.
=======================================================================
=== Currently Held Locks ==============================================
=======================================================================
===
=== <file> <line num> <function> <lock name> <lock addr> (times locked)
===
=== Thread ID: -1208542320 (do_devstate_changes started at [ 363]
devicestate.c ast_device_state_engine_init())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1215984752 (pbx_thread started at [ 2645] pbx.c
ast_pbx_start())
=== ---> Lock https://issues.asterisk.org/view.php?id=0 (channel.c): MUTEX 1556
ast_hangup (channel lock)
0x9be8648 (1)
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=1
(chan_local.c): MUTEX 581
local_hangup (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1215247472 (pbx_thread started at [ 2645] pbx.c
ast_pbx_start())
=== ---> Lock https://issues.asterisk.org/view.php?id=0 (channel.c): MUTEX 1704
ast_activate_generator (channel
lock) 0x9bc63f8 (1)
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=1
(chan_local.c): MUTEX 185
local_queue_frame (channel lock) 0x9be8648 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1217889392 (netconsole started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1219118192 (netconsole started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1215493232 (netconsole started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1217643632 (netconsole started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=======================================================================
While this is happening I can go into the asterisk CLI any type a command
like "core show channels" and see all active channels but if I want to run
another command after that like "sip show peers" I have to leave the CLI
and come back each time. Restarting asterisk cleans it up and removes the
lock call.
======================================================================
----------------------------------------------------------------------
(0121129) svnbot (reporter) - 2010-04-28 16:26
https://issues.asterisk.org/view.php?id=17185#c121129
----------------------------------------------------------------------
Repository: asterisk
Revision: 259899
_U branches/1.6.2/
U branches/1.6.2/channels/chan_local.c
U branches/1.6.2/main/channel.c
------------------------------------------------------------------------
r259899 | dvossel | 2010-04-28 16:26:30 -0500 (Wed, 28 Apr 2010) | 46
lines
Merged revisions 259870 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk
................
r259870 | dvossel | 2010-04-28 16:20:03 -0500 (Wed, 28 Apr 2010) | 39
lines
Merged revisions 259858 via svnmerge from
https://origsvn.digium.com/svn/asterisk/branches/1.4
........
r259858 | dvossel | 2010-04-28 16:16:03 -0500 (Wed, 28 Apr 2010) | 33
lines
resolves deadlocks in chan_local
Issue_1.
In the local_hangup() 3 locks must be held at the same time... pvt,
pvt->chan,
and pvt->owner. Proper deadlock avoidance is done when the channel to
hangup
is the outbound chan_local channel, but when it is not the outbound
channel we
have an issue... We attempt to do deadlock avoidance only on the tech
pvt, when
both the tech pvt and the pvt->owner are locked coming into that loop.
By
never giving up the pvt->owner channel deadlock avoidance is not
entirely possible.
This patch resolves that by doing deadlock avoidance on both the
pvt->owner and the pvt
when trying to get the pvt->chan lock.
Issue_2.
ast_prod() is used in ast_activate_generator() to queue a frame on the
channel
and make the channel's read function get called. This function is
used in
ast_activate_generator() while the channel is locked, which mean's the
channel
will have a lock both from the generator code and the frame_queue code
by the
time it gets to chan_local.c's local_queue_frame code...
local_queue_frame
contains some of the same crazy deadlock avoidance that local_hangup
requires,
and this recursive lock prevents that deadlock avoidance from
happening correctly.
This patch removes ast_prod() from the channel lock so only one lock
is held during
the local_queue_frame function.
(closes issue https://issues.asterisk.org/view.php?id=17185)
Reported by: schmoozecom
Patches:
issue_17185_v1.diff uploaded by dvossel (license 671)
issue_17185_v2.diff uploaded by dvossel (license 671)
Tested by: schmoozecom, GameGamer43
Review: https://reviewboard.asterisk.org/r/631/
........
................
------------------------------------------------------------------------
http://svn.digium.com/view/asterisk?view=rev&revision=259899
Issue History
Date Modified Username Field Change
======================================================================
2010-04-28 16:26 svnbot Checkin
2010-04-28 16:26 svnbot Note Added: 0121129
======================================================================
More information about the asterisk-bugs
mailing list