[asterisk-bugs] [Asterisk 0017888]: ast_add_hint deadlock in pbx.c MUTEX ast_hint_state_changed

[Asterisk Bug Tracker]

The following issue has been RESOLVED. 
====================================================================== 
https://issues.asterisk.org/view.php?id=17888 
====================================================================== 
Reported By:                zerohalo
Assigned To:                dvossel
====================================================================== 
Project:                    Asterisk
Issue ID:                   17888
Category:                   Core/PBX
Reproducibility:            random
Severity:                   major
Priority:                   normal
Status:                     resolved
Asterisk Version:           1.4.34 
JIRA:                       SWP-2111 
Regression:                 No 
Reviewboard Link:            
SVN Branch (only for SVN checkouts, not tarball releases): N/A 
SVN Revision (number only!):  
Request Review:              
Resolution:                 fixed
Fixed in Version:           
====================================================================== 
Date Submitted:             2010-08-19 13:33 CDT
Last Modified:              2010-09-10 15:03 CDT
====================================================================== 
Summary:                    ast_add_hint deadlock in pbx.c MUTEX
ast_hint_state_changed
Description: 
getting deadlock randomly, call processing stops - appears to happen when
the hint state changes when a call is hung up.

1.4.34 w/ patch from https://issues.asterisk.org/view.php?id=17643

'core show locks' and backtrace from ast_grab_core attached
======================================================================
Relationships       ID      Summary
----------------------------------------------------------------------
related to          0017643 [patch] dialplan reload deadlocks in as...
====================================================================== 

---------------------------------------------------------------------- 
 (0126836) svnbot (reporter) - 2010-09-10 15:03
 https://issues.asterisk.org/view.php?id=17888#c126836 
---------------------------------------------------------------------- 
Repository: asterisk
Revision: 286070

U   branches/1.4/channels/chan_sip.c

------------------------------------------------------------------------
r286070 | dvossel | 2010-09-10 15:03:51 -0500 (Fri, 10 Sep 2010) | 32
lines

Fixes sip extension state update DEADLOCK

PROBLEM:
In chan_sip, and all the other channel drivers, it is common for
us to hold the tech_pvt lock while we ask the Asterisk core about
an extension and context.  Every time we do this the locking
order becomes, (1. tech_pvt lock ---> 2. global context lock). In
chan_sip when a dialog subscribes to a hint, that locking order
is reversed in the extensionstate callback which will occur outside
of the channel_driver's monitor loop.  So, on an extension state
update we have (1. global context lock ----> 2. tech_pvt lock).

Typically when we have to do a reversed locking order like this
we'd just do some sort of deadlock avoidance to fix the problem...
That will not work here.  There are more locks involved here than
just the context and tech_pvt.  Those are the two that are colliding,
but it is impossible to give up the context lock because the global
hints list lock MUST be held as well and we can not give that lock
up during the extensionstate callback traversal... The locking order
for the context and hints are (1. global context lock ----> 2.
hints list lock).  Deadlock avoidance is not an option here.

SOLUTION:
The solution this patch implements is to queue the extension state updates
into a list and send the NOTIFY messages out during the do_monitor pvt
traversal.  This clears out the problem of having to hold the context
lock before the tech_pvt lock entirely.

(closes issue https://issues.asterisk.org/view.php?id=17888)
Reported by: zerohalo


------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=286070 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2010-09-10 15:03 svnbot         Note Added: 0126836                          
2010-09-10 15:03 svnbot         Status                   assigned => resolved
2010-09-10 15:03 svnbot         Resolution               open => fixed       
======================================================================