[asterisk-bugs] [JIRA] (ASTERISK-28894) new memory leak when updating 16.9.0 to 16.10.0

Mon May 18 14:07:25 CDT 2020

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=250845#comment-250845 ] 

Matthias Hensler commented on ASTERISK-28894:
---------------------------------------------

Some quick observation: there seems to be a leak at least in res_rtp_asterisk.c:
mem-summary-20200515-2303.txt:     20324 bytes in         65 allocations in file res_rtp_asterisk.c
mem-summary-20200516-0830.txt:    171676 bytes in        591 allocations in file res_rtp_asterisk.c
mem-summary-20200517-1109.txt:    754680 bytes in       4306 allocations in file res_rtp_asterisk.c
mem-summary-20200517-1129.txt:    636064 bytes in       3942 allocations in file res_rtp_asterisk.c
mem-summary-20200518-1053.txt:   9082322 bytes in      45111 allocations in file res_rtp_asterisk.c
mem-summary-20200518-1524.txt:  12640772 bytes in      76112 allocations in file res_rtp_asterisk.c
mem-summary-20200518-1706.txt:  11985152 bytes in      79534 allocations in file res_rtp_asterisk.c
mem-summary-20200518-1833.txt:   9742000 bytes in      73829 allocations in file res_rtp_asterisk.c
mem-summary-20200518-2055.txt:   9580584 bytes in      73412 allocations in file res_rtp_asterisk.c

stream.c and format_cap.c also increases:
mem-summary-20200515-2303.txt:      1440 bytes in         18 allocations in file stream.c
mem-summary-20200516-0830.txt:    223728 bytes in       2817 allocations in file stream.c
mem-summary-20200517-1109.txt:   1129648 bytes in      14175 allocations in file stream.c
mem-summary-20200517-1129.txt:   1141760 bytes in      14306 allocations in file stream.c
mem-summary-20200518-1053.txt:   4215416 bytes in      53559 allocations in file stream.c
mem-summary-20200518-1524.txt:   7841552 bytes in      98777 allocations in file stream.c
mem-summary-20200518-1706.txt:   8909888 bytes in     111854 allocations in file stream.c
mem-summary-20200518-1833.txt:   9143808 bytes in     114386 allocations in file stream.c
mem-summary-20200518-2055.txt:   9244928 bytes in     115616 allocations in file stream.c

mem-summary-20200515-2303.txt:   1388432 bytes in       7740 allocations in file format_cap.c
mem-summary-20200516-0830.txt:   1637272 bytes in      11531 allocations in file format_cap.c
mem-summary-20200517-1109.txt:   2919080 bytes in      29322 allocations in file format_cap.c
mem-summary-20200517-1129.txt:   1987976 bytes in      22160 allocations in file format_cap.c
mem-summary-20200518-1053.txt:   5151400 bytes in      71128 allocations in file format_cap.c
mem-summary-20200518-1524.txt:   6091200 bytes in     109867 allocations in file format_cap.c
mem-summary-20200518-1706.txt:   6312952 bytes in     121837 allocations in file format_cap.c
mem-summary-20200518-1833.txt:   6069936 bytes in     122639 allocations in file format_cap.c
mem-summary-20200518-2055.txt:   6046680 bytes in     123462 allocations in file format_cap.c

Please note: this is a vanilla build of 16.10.0, without(!) the patch from ASTERISK-28870 this time.

> new memory leak when updating 16.9.0 to 16.10.0
> -----------------------------------------------
>
>                 Key: ASTERISK-28894
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28894
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: .Release/Targets
>    Affects Versions: 16.10.0
>         Environment: Linux 64-bit, CentOS 7, Kernel 3.10.0-957.5.1.el7.x86_64
>            Reporter: Matthias Hensler
>            Assignee: Unassigned
>              Labels: fax
>         Attachments: memory-month.png, Memory_Summary.zip, memory.zip
>
>
> We currently follow the Asterisk 16 mainline, updating whenever a new version is available. After the recent update from 16.9.0 to 16.10.0 we see a steady increase in memory consumption, needing to restart the process regularly.
> Unfortunately I cannot pinpoint to a specific component, so I need some suggestions on how to hunt this down.
> I already lookup up know issues, but the only reported leak so far on 16.10.0 is ASTERISK-28870. However, I am pretty sure that we hit a different issue, as I recompiled our 16.10.0 with the mentioned patchset (commit 7fbfbe7) and still see the issue.
> Involved components in our setup are pretty straightforward, since we only route sip-calls while billing them. Beside that we are use fax, but no additional features like conferencing, voicemail, etc. The system routes a couple of 10,000 calls a day with a maximum of around 200 calls in parallel.
> Used components in detail:
> * chan_sip (our progress to migrate to res_pjsip is ongoing)
> * cdr_mysql (yes, deprecation status is known)
> * res_fax_spandsp (used for in- and outbound fax on g711a and t38)
> Used codecs are g711a and g722 only (plus t38 for fax).
> The dialplan uses an external generated extension.conf in ascii-format (no database, or ael/lua involved). All external scripting is done via FastAGI-calls from the dialplan.
> Beside that there a several connections to the AMI manager per minute to collect call stats.
> Asterisk is compiled from an unmodified sourcecode and installed under a separate prefix per version (in detail: "./configure --prefix=/usr/local/packages/asterisk-16.10.0-7fbfbe7-issue28870 --disable-xmldoc --with-jansson-bundled", followed by a "make menuconfig" to enable cdr_mysql and opus-codec translater (which is not used at the moment) and then "make" and "make install").
> When looking at the memory usage the Asteriskprocess will increase by around 200-300 MBytes per day. The increase is only noticeable in the daytime when we have to most calls, in the nighttime were calls are few there is no increase. Therefore I would rule out any issues with the manager AMI, since connections are constant throughout the day.
> I apologise for just reporting  the memory leak without giving more hints at to which component might be affected. I will study the changeset from 16.9.0 to 16.10.0 more carefully if something spots the eye. If you need more details or give me any more hints on how to debug further I will happily provide (currently the process is still in a leak state and I have no pressure to restart within the next days. So if any CLI-commands are needed for checking some detailed status, just ask).

--
This message was sent by Atlassian JIRA
(v6.2#6252)