[asterisk-bugs] [Asterisk 0013409]: [patch] Huge memory leak because memory of channel cdr struct is never returned

[Asterisk Bug Tracker]

A NOTE has been added to this issue. 
====================================================================== 
http://bugs.digium.com/view.php?id=13409 
====================================================================== 
Reported By:                tomaso
Assigned To:                murf
====================================================================== 
Project:                    Asterisk
Issue ID:                   13409
Category:                   Core/Channels
Reproducibility:            always
Severity:                   block
Priority:                   normal
Status:                     feedback
Asterisk Version:           SVN 
SVN Branch (only for SVN checkouts, not tarball releases): 1.6.0 
SVN Revision (number only!): 137818 
Disclaimer on File?:        N/A 
Request Review:              
====================================================================== 
Date Submitted:             2008-09-01 09:48 CDT
Last Modified:              2008-09-09 10:05 CDT
====================================================================== 
Summary:                    [patch] Huge memory leak because memory of channel
cdr struct is never returned
Description: 
After two days of stress testing by making lots of calls across sip and
dahdi channels the asterisk process memory reached dizzy values: VSZ=3,2GB,
RSS=1,6GB before asterisk stucked completely (even the RAM of our server
(2GB) is finite ;-) ).

Actually this problem is not a question of load, but appears for each
single call.

Using valgrind the reason for that was quickly found: The memory of the
channel cdr struct (chan->cdr) is never returned, not for sip, not for
dahdi channels, when a channel is cleared.

Reproduce:
a.f.a.p. default configuration (modules.conf, etc.)
Make lots of calls and see ps's VSZ and RSS values.

Interested in a patch ? Or is someone revise this anyway?

======================================================================
Relationships       ID      Summary
----------------------------------------------------------------------
related to          0013235 Memory leak in Asterisk 1.4 and Trunk
related to          0013444 After update from 140415 to 141991 get ...
====================================================================== 

---------------------------------------------------------------------- 
 (0092247) murf (administrator) - 2008-09-09 10:05
 http://bugs.digium.com/view.php?id=13409#c92247 
---------------------------------------------------------------------- 
Many thanks to Russell. The patch looks good. It should prevent the crash,
and is a light year ahead of I earlier proposed.

tomaso-- 

You don't have to worry about the cdr issue; your leak is much more
important right now.

so, the plug we installed did not affect you? Very interesting!
And you still have channels stuck in the Rx: BYE state?  Then here is how
we
find it. If 100% of your channels end up in the Rx: BYE state, then it
will be
easy: 

1. "sip set debug", while you are processing calls.If you are processing a
lot of channels, you will not have to do this for long. 
2. Collect all the output, and copy it into a file. 
3. Wait a liitle more than 32 seconds, because that's how long the
destruction callback is scheduled for; do a "sip show channels", and find
one channel that went into the Rx:BYE state while you had "sip set debug"
on, and is still in that state after 32 seconds. 
4. The channel name is the dialog name. Find that dialog id in your file,
and find just those transactions that involve this dialog id. The text
above each such transaction is sometimes also revealing. Hopefully, this
will reveal to us, what is happening in the driver that leads to the
channel getting stuck.
5. Attach your filtered output to this bug.


If it is so, that I misunderstood your bug, and there are still noticeable
leaks, but not the dead-channels, then use "make menuselect" to set
compiler option "MALLOC_DEBUG". Recompile your source. start asterisk.
Enter the command "memory show allocations" and collect the output into a
file.
Then run a couple hundred calls. then "memory show allocations" again, and
collect the output into another file. Diff the two files. share with us the
result. 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2008-09-09 10:05 murf           Note Added: 0092247                          
======================================================================