[asterisk-bugs] [Asterisk 0017699]: user get unreachable after some minutes, deadlock in ao2_lock

[Asterisk Bug Tracker]

A NOTE has been added to this issue. 
====================================================================== 
https://issues.asterisk.org/view.php?id=17699 
====================================================================== 
Reported By:                schmidts
Assigned To:                
====================================================================== 
Project:                    Asterisk
Issue ID:                   17699
Category:                   Channels/chan_sip/General
Reproducibility:            sometimes
Severity:                   minor
Priority:                   normal
Status:                     new
Asterisk Version:           1.6.2.10 
JIRA:                        
Regression:                 No 
Reviewboard Link:            
SVN Branch (only for SVN checkouts, not tarball releases): N/A 
SVN Revision (number only!):  
Request Review:              
====================================================================== 
Date Submitted:             2010-07-24 17:30 CDT
Last Modified:              2010-07-26 08:32 CDT
====================================================================== 
Summary:                    user get unreachable after some minutes, deadlock in
ao2_lock
Description: 
i´ve tried the second time to upgrade from 1.2.40 to 1.6.2.9 / 10 and
after 10 to 15 minutes it looks like asterisk couldnt send any sip packets
out.
i thought that the last revision r278465 could also solve this problem so
i´ve tried 1.6.2.10 today but i got the same problem.

i have around 3100 peers in static config files and normally around 2100
are registered and reachable in asterisk. there are also around 1300 hints
in the extensions.conf and more than 1300 subcribes on that hint.
when i restart asterisk, the peers get the qualify packet and are
reachable but after some time, i see more and more of these messages:
 == Extension Changed userB[outcust] new state Idle for Notify User userA
(queued)
also the phones cant reach asterisk any more and show they are offline,
and in asterisk i can see them as unreachable.
with 1.6.2.9 after 1 hour only 100 phones has keep rechable and in
1.6.2.10 only 1000 after 30 mins.

i can also see that their are many (>500) open sip channels which i dont
see in 1.2.40
this server is a hosted pbx solution for our customers and when i´ve
tested this there is no phone traffic on the server just registrations and
subscribes.
with load this server handles > 200 concurrent calls without any problems.
====================================================================== 

---------------------------------------------------------------------- 
 (0125018) schmidts (reporter) - 2010-07-26 08:32
 https://issues.asterisk.org/view.php?id=17699#c125018 
---------------------------------------------------------------------- 
i can reproduce this problem with the following setup:

server A with asterisk 1.6.2.10 and sip.conf of 2500 friends
server B with asterisk 1.6.2.10 which register these 2500 peers to Server
A

when server A runs and i start asterisk on server B i see that all of the
2500 users come online and stay reachable until i do a sip reload on server
B. Then users starting getting unreachable until only 500 of the 2500 are
online. until server B finish the registration server a also held some
locks. When its finish everything again is wealthy.

i´ve done 4 backtraces, first when all is ok, second and third during the
problem and fourth after everything is ok, again.

have a look at the attached file, this thread is the only one which
changes and also has a look situation. 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2010-07-26 08:32 schmidts       Note Added: 0125018                          
======================================================================