[asterisk-bugs] [Asterisk 0015973]: Huge memory consumption after few hours of load

[Asterisk Bug Tracker]

A NOTE has been added to this issue. 
====================================================================== 
https://issues.asterisk.org/view.php?id=15973 
====================================================================== 
Reported By:                atis
Assigned To:                
====================================================================== 
Project:                    Asterisk
Issue ID:                   15973
Category:                   Core/General
Reproducibility:            always
Severity:                   major
Priority:                   normal
Status:                     acknowledged
Asterisk Version:           1.6.1.5 
JIRA:                        
Regression:                 No 
SVN Branch (only for SVN checkouts, not tarball releases): N/A 
SVN Revision (number only!):  
Request Review:              
====================================================================== 
Date Submitted:             2009-09-27 17:14 CDT
Last Modified:              2009-10-01 19:37 CDT
====================================================================== 
Summary:                    Huge memory consumption after few hours of load
Description: 
I'm doing load tests on Asterisk 1.6.1.5 and after few hours of load, it
takes 80% of ram (6GB installed), so it becomes unable to fork itself (when
using System() dialplan app), and creating core dump takes an hour. Core
dumps are in size of 9 Gigabytes.

Memory summary doesn't show anytingh of much usage:

80469516 bytes allocated (8800 in caches) in 71421 allocations

while top shows:

Mem:   6055876k total,  6009316k used,    46560k free,    19220k buffers
Swap:  4096564k total,  4096548k used,       16k free,   249980k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15870 asterisk  10 -10 9780m 4.8g 3936 S    6 83.1 505:00.32 asterisk

====================================================================== 

---------------------------------------------------------------------- 
 (0111768) atis (reporter) - 2009-10-01 19:37
 https://issues.asterisk.org/view.php?id=15973#c111768 
---------------------------------------------------------------------- 
I have reproduced memory usage under valgrind, and got leak summary
(attached valgrind.2.txt), which shows most memory lost within:

 4,220,212,480 bytes in 6,594,082 blocks are possibly lost in loss record
1,960 of 1,960
    at 0x4A05E7C: malloc (vg_replace_malloc.c:195)
    by 0x43F882: __ast_alloc_region (astmm.c:114)
    by 0x43F81D: __ast_calloc (astmm.c:204)
    by 0x4C564F: ast_log (logger.c:1138)
    by 0x458F3C: channel_find_locked (channel.c:1233)
    by 0x459056: ast_channel_walk_locked (channel.c:1267)
    by 0x46D0C3: handle_chanlist (cli.c:685)
    by 0x470CD4: ast_cli_command (cli.c:1888)
    by 0x470F3F: ast_cli_command_multiple (cli.c:1921)
    by 0x4370FA: netconsole (asterisk.c:1039)
    by 0x541B12: dummy_start (utils.c:968)
    by 0x3A42A06406: start_thread (pthread_create.c:297)

However this doesn't seem to be the issue I was experiencing initially, as
I added periodic calls to "core show channels" only when debugging in
valgrind to monitor calls status. 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2009-10-01 19:37 atis           Note Added: 0111768                          
======================================================================