[asterisk-bugs] [Asterisk 0017185]: [patch] [regression] Using Local channels with queues causes deadlocks

Asterisk Bug Tracker noreply at bugs.digium.com
Wed Apr 28 16:33:18 CDT 2010


A NOTE has been added to this issue. 
====================================================================== 
https://issues.asterisk.org/view.php?id=17185 
====================================================================== 
Reported By:                schmoozecom
Assigned To:                dvossel
====================================================================== 
Project:                    Asterisk
Issue ID:                   17185
Category:                   Channels/chan_local
Reproducibility:            sometimes
Severity:                   block
Priority:                   normal
Status:                     closed
Target Version:             1.4.31
Asterisk Version:           1.4.30 
JIRA:                       SWP-1295 
Regression:                 No 
Reviewboard Link:            
SVN Branch (only for SVN checkouts, not tarball releases): N/A 
SVN Revision (number only!):  
Request Review:              
Resolution:                 fixed
Fixed in Version:           
====================================================================== 
Date Submitted:             2010-04-14 16:23 CDT
Last Modified:              2010-04-28 16:33 CDT
====================================================================== 
Summary:                    [patch] [regression] Using Local channels with
queues causes deadlocks
Description: 
Since upgrading to asterisk 1.4.30 we are seeing calls that get answered in
the queue still show as calls waiting in the queue.  We have seen this on 2
systems now and downgrading to 1.4.29 solves the issue.  When looking at
"Show Queue" it will show a dahdi channel as still waiting in the queue
even though the call has been answered and hung up.

Below is a print out of "core show locks" when asterisk is in this state.

=======================================================================
=== Currently Held Locks ==============================================
=======================================================================
===
=== <file> <line num> <function> <lock name> <lock addr> (times locked)
===
=== Thread ID: -1208542320 (do_devstate_changes  started at [  363]
devicestate.c ast_device_state_engine_init())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1215984752 (pbx_thread           started at [ 2645] pbx.c
ast_pbx_start())
=== ---> Lock https://issues.asterisk.org/view.php?id=0 (channel.c): MUTEX 1556
ast_hangup (channel lock)
0x9be8648 (1)
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=1
(chan_local.c): MUTEX 581
local_hangup (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1215247472 (pbx_thread           started at [ 2645] pbx.c
ast_pbx_start())
=== ---> Lock https://issues.asterisk.org/view.php?id=0 (channel.c): MUTEX 1704
ast_activate_generator (channel
lock) 0x9bc63f8 (1)
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=1
(chan_local.c): MUTEX 185
local_queue_frame (channel lock) 0x9be8648 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1217889392 (netconsole           started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1219118192 (netconsole           started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1215493232 (netconsole           started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=== Thread ID: -1217643632 (netconsole           started at [ 1036]
asterisk.c listener())
=== ---> Tried and failed to get Lock https://issues.asterisk.org/view.php?id=0
(channel.c): MUTEX 1130
channel_find_locked (channel lock) 0x9bc63f8 (0)
=== -------------------------------------------------------------------
===
=======================================================================


While this is happening I can go into the asterisk CLI any type a command
like "core show channels" and see all active channels but if I want to run
another command after that like "sip show peers" I have to leave the CLI
and come back each time.  Restarting asterisk cleans it up and removes the
lock call.
====================================================================== 

---------------------------------------------------------------------- 
 (0121130) svnbot (reporter) - 2010-04-28 16:33
 https://issues.asterisk.org/view.php?id=17185#c121130 
---------------------------------------------------------------------- 
Repository: asterisk
Revision: 259930

_U  branches/1.6.1/
U   branches/1.6.1/channels/chan_local.c
U   branches/1.6.1/main/channel.c

------------------------------------------------------------------------
r259930 | dvossel | 2010-04-28 16:33:17 -0500 (Wed, 28 Apr 2010) | 46
lines

Merged revisions 259870 via svnmerge from 
https://origsvn.digium.com/svn/asterisk/trunk

................
  r259870 | dvossel | 2010-04-28 16:20:03 -0500 (Wed, 28 Apr 2010) | 39
lines
  
  Merged revisions 259858 via svnmerge from 
  https://origsvn.digium.com/svn/asterisk/branches/1.4
  
  ........
    r259858 | dvossel | 2010-04-28 16:16:03 -0500 (Wed, 28 Apr 2010) | 33
lines
    
    resolves deadlocks in chan_local
    
    Issue_1.
    In the local_hangup() 3 locks must be held at the same time... pvt,
pvt->chan,
    and pvt->owner.  Proper deadlock avoidance is done when the channel to
hangup
    is the outbound chan_local channel, but when it is not the outbound
channel we
    have an issue... We attempt to do deadlock avoidance only on the tech
pvt, when
    both the tech pvt and the pvt->owner are locked coming into that loop.
 By
    never giving up the pvt->owner channel deadlock avoidance is not
entirely possible.
    This patch resolves that by doing deadlock avoidance on both the
pvt->owner and the pvt
    when trying to get the pvt->chan lock.
    
    Issue_2.
    ast_prod() is used in ast_activate_generator() to queue a frame on the
channel
    and make the channel's read function get called.  This function is
used in
    ast_activate_generator() while the channel is locked, which mean's the
channel
    will have a lock both from the generator code and the frame_queue code
by the
    time it gets to chan_local.c's local_queue_frame code...
local_queue_frame
    contains some of the same crazy deadlock avoidance that local_hangup
requires,
    and this recursive lock prevents that deadlock avoidance from
happening correctly.
    This patch removes ast_prod() from the channel lock so only one lock
is held during
    the local_queue_frame function.
    
    (closes issue https://issues.asterisk.org/view.php?id=17185)
    Reported by: schmoozecom
    Patches:
          issue_17185_v1.diff uploaded by dvossel (license 671)
          issue_17185_v2.diff uploaded by dvossel (license 671)
    Tested by: schmoozecom, GameGamer43
    
    Review: https://reviewboard.asterisk.org/r/631/
  ........
................

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=259930 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2010-04-28 16:33 svnbot         Checkin                                      
2010-04-28 16:33 svnbot         Note Added: 0121130                          
======================================================================




More information about the asterisk-bugs mailing list