[asterisk-dev] memory leaks

Thu Jan 26 23:09:48 MST 2006

On Thu, 2006-01-26 at 22:48 -0600, Tilghman Lesher wrote:
> On Thursday 26 January 2006 19:31, Beau Hargis wrote:
> > I am seeing what look like memory leaks. I am using the 1.2.x
> > branch updated as of a couple of hours ago. I saw really, really
> > bad memory leaks with res_perl loaded in and being used. I
> > eliminated the res_perl stuff and use AGI instead, but the memory
> > leaks still seemed to continue. I put it into valgrind and it kept
> > reporting:
> >
> > ==3441== Warning: invalid file descriptor 1017 in syscall close()
> > ==3441== Warning: invalid file descriptor 1018 in syscall close()
> > ==3441== Warning: invalid file descriptor 1019 in syscall close()
> > ==3441== Warning: invalid file descriptor 1020 in syscall close()
> > ==3441== Warning: invalid file descriptor 1021 in syscall close()
> > ==3441== Warning: invalid file descriptor 1022 in syscall close()
> > ==3441== Warning: invalid file descriptor 1023 in syscall close()
> > ==3445== Warning: invalid file descriptor 1014 in syscall close()
> > ==3445== Warning: invalid file descriptor 1015 in syscall close()
> > ==3445== Warning: invalid file descriptor 1016 in syscall close()
> >
> > over and over several times during every call after each instance
> > of AGI being called. The summary at the end did say quite a bit
> > about memory leaks being present.
> 
> Those messages are completely unrelated to any memory leaks.  Those
> messages relate to a forked process needing to close all open file
> descriptors before execing the AGI program.

When I let it run for a couple of hours today, I got a summary detailing
megabytes of lost memory.

The output I got from a quick run on a production system taking a number
of calls:

==30319== Thread 5:
==30319== Invalid read of size 8
==30319==    at 0x4BF3A0F: ??? (res_musiconhold.c:512)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319==    by 0x3BAEDC5B42: clone (in /lib64/tls/libc-2.3.4.so)
==30319==  Address 0x4A466A0 is 66,184 bytes inside a block of size
66,208 free'd
==30319==    at 0x49055DD: free (vg_replace_malloc.c:235)
==30319==    by 0x4BF27E6: ??? (res_musiconhold.c:169)
==30319==    by 0x4BF5740: ??? (res_musiconhold.c:1085)
==30319==    by 0x46D50D: ast_run_atexits (asterisk.c:824)
==30319==    by 0x46D7D1: quit_handler (asterisk.c:889)
==30319==    by 0x46DA2B: __quit_handler (asterisk.c:938)
==30319==    by 0x3BAED2E40F: (within /lib64/tls/libc-2.3.4.so)
==30319== 
==30319== Invalid read of size 4
==30319==    at 0x4BF3812: ??? (res_musiconhold.c:486)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319==    by 0x3BAEDC5B42: clone (in /lib64/tls/libc-2.3.4.so)
==30319==  Address 0x4A466A8 is 66,192 bytes inside a block of size
66,208 free'd
==30319==    at 0x49055DD: free (vg_replace_malloc.c:235)
==30319==    by 0x4BF27E6: ??? (res_musiconhold.c:169)
==30319==    by 0x4BF5740: ??? (res_musiconhold.c:1085)
==30319==    by 0x46D50D: ast_run_atexits (asterisk.c:824)
==30319==    by 0x46D7D1: quit_handler (asterisk.c:889)
==30319==    by 0x46DA2B: __quit_handler (asterisk.c:938)
==30319==    by 0x3BAED2E40F: (within /lib64/tls/libc-2.3.4.so)
==30319== 
==30319== Invalid read of size 4
==30319==    at 0x4BF386F: ??? (res_musiconhold.c:493)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319==    by 0x3BAEDC5B42: clone (in /lib64/tls/libc-2.3.4.so)
==30319==  Address 0x4A466AC is 66,196 bytes inside a block of size
66,208 free'd
==30319==    at 0x49055DD: free (vg_replace_malloc.c:235)
==30319==    by 0x4BF27E6: ??? (res_musiconhold.c:169)
==30319==    by 0x4BF5740: ??? (res_musiconhold.c:1085)
==30319==    by 0x46D50D: ast_run_atexits (asterisk.c:824)
==30319==    by 0x46D7D1: quit_handler (asterisk.c:889)
==30319==    by 0x46DA2B: __quit_handler (asterisk.c:938)
==30319==    by 0x3BAED2E40F: (within /lib64/tls/libc-2.3.4.so)
==30319== 
==30319== Invalid read of size 4
==30319==    at 0x4BF3883: ??? (res_musiconhold.c:495)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319==    by 0x3BAEDC5B42: clone (in /lib64/tls/libc-2.3.4.so)
==30319==  Address 0x4A466AC is 66,196 bytes inside a block of size
66,208 free'd
==30319==    at 0x49055DD: free (vg_replace_malloc.c:235)
==30319==    by 0x4BF27E6: ??? (res_musiconhold.c:169)
==30319==    by 0x4BF5740: ??? (res_musiconhold.c:1085)
==30319==    by 0x46D50D: ast_run_atexits (asterisk.c:824)
==30319==    by 0x46D7D1: quit_handler (asterisk.c:889)
==30319==    by 0x46DA2B: __quit_handler (asterisk.c:938)
==30319==    by 0x3BAED2E40F: (within /lib64/tls/libc-2.3.4.so)
==30319== 
==30319== ERROR SUMMARY: 248 errors from 6 contexts (suppressed: 242
from 1)
==30319== malloc/free: in use at exit: 1,493,787 bytes in 4,255 blocks.
==30319== malloc/free: 8,096 allocs, 3,841 frees, 2,128,087 bytes
allocated.
==30319== For counts of detected errors, rerun with: -v
==30319== searching for pointers to 4,255 not-freed blocks.
==30319== checked 12,754,848 bytes.
==30319== 
==30319== Thread 1:
==30319== 
==30319== 18 (16 direct, 2 indirect) bytes in 1 blocks are definitely
lost in loss record 25 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x486FD4: ast_yylex (ast_expr2.fl:76)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 18 (16 direct, 2 indirect) bytes in 1 blocks are definitely
lost in loss record 26 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x487080: ast_yylex (ast_expr2.fl:77)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 72 (64 direct, 8 indirect) bytes in 4 blocks are definitely
lost in loss record 228 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x486514: ast_yylex (ast_expr2.fl:60)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 80 bytes in 5 blocks are definitely lost in loss record 235 of
348
==30319==    at 0x4904A86: malloc (vg_replace_malloc.c:149)
==30319==    by 0x484B6F: make_integer (ast_expr2.y:235)
==30319==    by 0x4856F4: op_plus (ast_expr2.y:639)
==30319==    by 0x4840F2: ast_yyparse (ast_expr2.y:196)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440671: pbx_substitute_variables_helper_full
(pbx.c:1568)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319== 
==30319== 
==30319== 80 bytes in 5 blocks are definitely lost in loss record 236 of
348
==30319==    at 0x4904A86: malloc (vg_replace_malloc.c:149)
==30319==    by 0x484B6F: make_integer (ast_expr2.y:235)
==30319==    by 0x4852C9: op_lt (ast_expr2.y:499)
==30319==    by 0x483F7A: ast_yyparse (ast_expr2.y:184)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319== 
==30319== 
==30319== 80 bytes in 5 blocks are definitely lost in loss record 237 of
348
==30319==    at 0x4904A86: malloc (vg_replace_malloc.c:149)
==30319==    by 0x484B6F: make_integer (ast_expr2.y:235)
==30319==    by 0x485219: op_gt (ast_expr2.y:479)
==30319==    by 0x483F1C: ast_yyparse (ast_expr2.y:181)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319== 
==30319== 
==30319== 90 (80 direct, 10 indirect) bytes in 5 blocks are definitely
lost in loss record 239 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x486718: ast_yylex (ast_expr2.fl:63)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 90 (80 direct, 10 indirect) bytes in 5 blocks are definitely
lost in loss record 240 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x4867C4: ast_yylex (ast_expr2.fl:64)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 90 (80 direct, 10 indirect) bytes in 5 blocks are definitely
lost in loss record 241 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x486A74: ast_yylex (ast_expr2.fl:68)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440671: pbx_substitute_variables_helper_full
(pbx.c:1568)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319== 
==30319== 
==30319== 348 (256 direct, 92 indirect) bytes in 16 blocks are
definitely lost in loss record 292 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x487284: ast_yylex (ast_expr2.fl:86)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 432 bytes in 27 blocks are definitely lost in loss record 293
of 348
==30319==    at 0x4904A86: malloc (vg_replace_malloc.c:149)
==30319==    by 0x484B6F: make_integer (ast_expr2.y:235)
==30319==    by 0x48516A: op_eq (ast_expr2.y:459)
==30319==    by 0x483EBE: ast_yyparse (ast_expr2.y:178)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319== 
==30319== 
==30319== 486 (432 direct, 54 indirect) bytes in 27 blocks are
definitely lost in loss record 297 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x48666C: ast_yylex (ast_expr2.fl:62)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 2,358 (2,096 direct, 262 indirect) bytes in 131 blocks are
definitely lost in loss record 323 of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x4871D8: ast_yylex (ast_expr2.fl:84)
==30319==    by 0x483AC4: ast_yyparse (ast_expr2.c:1149)
==30319==    by 0x488F6E: ast_expr (ast_expr2.fl:112)
==30319==    by 0x44069C: pbx_substitute_variables_helper_full
(pbx.c:1574)
==30319==    by 0x440756: pbx_substitute_variables_helper (pbx.c:1588)
==30319==    by 0x440847: pbx_substitute_variables (pbx.c:1606)
==30319==    by 0x440AA0: pbx_extension_helper (pbx.c:1663)
==30319==    by 0x44213E: ast_spawn_extension (pbx.c:2218)
==30319==    by 0x442586: __ast_pbx_run (pbx.c:2284)
==30319==    by 0x443209: pbx_thread (pbx.c:2505)
==30319==    by 0x3BAF6060A9: start_thread
(in /lib64/tls/libpthread-2.3.4.so)
==30319== 
==30319== 
==30319== 3,128 bytes in 23 blocks are possibly lost in loss record 327
of 348
==30319==    at 0x4905DA7: calloc (vg_replace_malloc.c:279)
==30319==    by 0x3BAEB0D1B2: _dl_allocate_tls (in /lib64/ld-2.3.4.so)
==30319==    by 0x3BAF606633: pthread_create@@GLIBC_2.2.5
(in /lib64/tls/libpthread-2.3.4.so)
==30319==    by 0x47F007: ast_pthread_create_stack (utils.c:512)
==30319==    by 0x47E990: test_for_thread_safety (utils.c:260)
==30319==    by 0x471234: main (asterisk.c:2277)
==30319== 
==30319== LEAK SUMMARY:
==30319==    definitely lost: 3,792 bytes in 237 blocks.
==30319==    indirectly lost: 450 bytes in 112 blocks.
==30319==      possibly lost: 3,128 bytes in 23 blocks.
==30319==    still reachable: 1,486,417 bytes in 3,883 blocks.
==30319==         suppressed: 0 bytes in 0 blocks.
==30319== Reachable blocks (those to which a pointer was found) are not
shown.
==30319== To see them, rerun with: --show-reachable=yes