[asterisk-bugs] [Asterisk 0018165]: [patch] hint state changes deadlock problem

Thu Nov 11 13:42:09 CST 2010

A NOTE has been added to this issue. 
====================================================================== 
https://issues.asterisk.org/view.php?id=18165 
====================================================================== 
Reported By:                antonio
Assigned To:                jpeeler
====================================================================== 
Project:                    Asterisk
Issue ID:                   18165
Category:                   Core/PBX
Reproducibility:            have not tried
Severity:                   major
Priority:                   normal
Status:                     closed
Asterisk Version:           1.6.2.13 
JIRA:                       SWP-2495 
Regression:                 No 
Reviewboard Link:            
SVN Branch (only for SVN checkouts, not tarball releases): N/A 
SVN Revision (number only!):  
Request Review:              
Resolution:                 fixed
Fixed in Version:           
====================================================================== 
Date Submitted:             2010-10-19 13:45 CDT
Last Modified:              2010-11-11 13:42 CST
====================================================================== 
Summary:                    [patch] hint state changes deadlock problem
Description: 
What happens is there are two threads that lock resources in a different
order causing a deadlock.

The threads are:

1. from taskprocessor handle_statechange() is called. This one locks
ast_rdlock_contexts, and then eventually call cb_extensionstate() which
immediately does a sip_pvt_lock()

2. do_monitor calls sipsock_read() which calls handle_request_do() which
calls find_call(), which does a sip_pvt_lock() too. The incoming request is
a subscription, so it calls ast_get_hint(), which calls
ast_hint_extension() which calls ast_rdlock_contexts(). Deadlock.

======================================================================
Relationships       ID      Summary
----------------------------------------------------------------------
has duplicate       0018205 PBX hangup
has duplicate       0018260 Asterisk stop processing any SIP request
====================================================================== 

---------------------------------------------------------------------- 
 (0128792) svnbot (reporter) - 2010-11-11 13:42
 https://issues.asterisk.org/view.php?id=18165#c128792 
---------------------------------------------------------------------- 
Repository: asterisk
Revision: 294640

_U  branches/1.8/
U   branches/1.8/include/asterisk.h
U   branches/1.8/main/asterisk.c
U   branches/1.8/main/pbx.c

------------------------------------------------------------------------
r294640 | jpeeler | 2010-11-11 13:42:07 -0600 (Thu, 11 Nov 2010) | 60
lines

Merged revisions 294639 via svnmerge from 
https://origsvn.digium.com/svn/asterisk/branches/1.6.2

................
  r294639 | jpeeler | 2010-11-11 13:31:00 -0600 (Thu, 11 Nov 2010) | 53
lines

  Merged revisions 294384 via svnmerge from 
  https://origsvn.digium.com/svn/asterisk/branches/1.4

  ........
    r294384 | jpeeler | 2010-11-09 11:37:59 -0600 (Tue, 09 Nov 2010) | 47
lines

    Fix a deadlock in device state change processing.

    Copied from some notes from the original author (Russell):

    Deadlock scenario:
    Thread 1: device state change thread
      Holds - rdlock on contexts
      Holds - hints lock
      Waiting on channels container lock

    Thread 2: SIP monitor thread
      Holds the "iflock"
      Holds a sip_pvt lock
      Holds channel container lock
      Waiting for a channel lock

    Thread 3: A channel thread (chan_local in this case)
      Holds 2 channel locks acquired within app_dial
      Holds a 3rd channel lock it got inside of chan_local
      Holds a local_pvt lock
      Waiting on a rdlock of the contexts lock

    A bunch of other threads waiting on a wrlock of the contexts lock

    To address this deadlock, some locking order rules must be put in
place and
    enforced. Existing relevant rules:

    1) channel lock before a pvt lock
    2) contexts lock before hints lock
    3) channels container before a channel

    What's missing is some enforcement of the order when you involve more
than any
    two. To fix this problem, I put in some code that ensures that (at
least in the
    code paths involved in this bug) the locks in (3) come before the
locks in (2).
    To change the operation of thread 1 to comply, I converted the storage
of hints
    to an astobj2 container. This allows processing of hints without
holding the
    hints container lock. So, in the code path that led to thread 1's
state, it no
    longer holds either the contexts or hints lock while it attempts to
lock the
    channels container.

    (closes issue https://issues.asterisk.org/view.php?id=18165)
    Reported by: antonio

    ABE-2583
  ........
................

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=294640 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2010-11-11 13:42 svnbot         Checkin                                      
2010-11-11 13:42 svnbot         Note Added: 0128792                          
======================================================================