[asterisk-bugs] [JIRA] (ASTERISK-28894) new memory leak when updating 16.9.0 to 16.10.0

[Ross Beer (JIRA)]

     [ https://issues.asterisk.org/jira/browse/ASTERISK-28894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ross Beer updated ASTERISK-28894:
---------------------------------

    Attachment: Memory_Summary.zip

I am attaching the output from a server that is doing very little work with calls passing between two sip endpoints.

I have done a few calls to queues also as our queue server is currently using over 10GB of memory just running Asterisk. It's only been up for around 12 hours. An earlier asterisk version is also running queues and runs at around 4-5GB and has been up for months processing the same number of calls.

> new memory leak when updating 16.9.0 to 16.10.0
> -----------------------------------------------
>
>                 Key: ASTERISK-28894
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28894
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: .Release/Targets
>    Affects Versions: 16.10.0
>         Environment: Linux 64-bit, CentOS 7, Kernel 3.10.0-957.5.1.el7.x86_64
>            Reporter: Matthias Hensler
>            Assignee: Matthias Hensler
>              Labels: fax
>         Attachments: memory-month.png, Memory_Summary.zip
>
>
> We currently follow the Asterisk 16 mainline, updating whenever a new version is available. After the recent update from 16.9.0 to 16.10.0 we see a steady increase in memory consumption, needing to restart the process regularly.
> Unfortunately I cannot pinpoint to a specific component, so I need some suggestions on how to hunt this down.
> I already lookup up know issues, but the only reported leak so far on 16.10.0 is ASTERISK-28870. However, I am pretty sure that we hit a different issue, as I recompiled our 16.10.0 with the mentioned patchset (commit 7fbfbe7) and still see the issue.
> Involved components in our setup are pretty straightforward, since we only route sip-calls while billing them. Beside that we are use fax, but no additional features like conferencing, voicemail, etc. The system routes a couple of 10,000 calls a day with a maximum of around 200 calls in parallel.
> Used components in detail:
> * chan_sip (our progress to migrate to res_pjsip is ongoing)
> * cdr_mysql (yes, deprecation status is known)
> * res_fax_spandsp (used for in- and outbound fax on g711a and t38)
> Used codecs are g711a and g722 only (plus t38 for fax).
> The dialplan uses an external generated extension.conf in ascii-format (no database, or ael/lua involved). All external scripting is done via FastAGI-calls from the dialplan.
> Beside that there a several connections to the AMI manager per minute to collect call stats.
> Asterisk is compiled from an unmodified sourcecode and installed under a separate prefix per version (in detail: "./configure --prefix=/usr/local/packages/asterisk-16.10.0-7fbfbe7-issue28870 --disable-xmldoc --with-jansson-bundled", followed by a "make menuconfig" to enable cdr_mysql and opus-codec translater (which is not used at the moment) and then "make" and "make install").
> When looking at the memory usage the Asteriskprocess will increase by around 200-300 MBytes per day. The increase is only noticeable in the daytime when we have to most calls, in the nighttime were calls are few there is no increase. Therefore I would rule out any issues with the manager AMI, since connections are constant throughout the day.
> I apologise for just reporting  the memory leak without giving more hints at to which component might be affected. I will study the changeset from 16.9.0 to 16.10.0 more carefully if something spots the eye. If you need more details or give me any more hints on how to debug further I will happily provide (currently the process is still in a leak state and I have no pressure to restart within the next days. So if any CLI-commands are needed for checking some detailed status, just ask).



--
This message was sent by Atlassian JIRA
(v6.2#6252)