[asterisk-bugs] [JIRA] (ASTERISK-25905) Memory leak during perf testing

[Richard Mudgett (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-25905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=230261#comment-230261 ] 

Richard Mudgett commented on ASTERISK-25905:
--------------------------------------------

If you still have MALLOC_DEBUG enabled, you should disable it for performance testing.  MALLOC_DEBUG writes over every freed block of memory with a fill pattern and it holds onto the block for awhile in a deferred free pool.  Access to the deferred free pool serializes freeing memory.  The necessary actions of MALLOC_DEBUG reduce performance because Asterisk allocates and frees a lot of memory blocks from a lot of threads.

Unless you disable CDRs in the {{cdr.conf}} file, CDRs are generated.  If you don't have a backend enabled then the generated records have nowhere to go but the bit bucket.  The CDR engine's task processor queue is the raw events from calls monitored by the engine to generate CDRs.  There can be only one thread processing that queue to generate CDRs.

I don't think CDRs are a good fit for your situation because you wind up with a combinatorial explosion of CDRs to represent every pairing in a conference.  I suggest you look at CEL to create your own records of conferences.

https://wiki.asterisk.org/wiki/display/AST/Asterisk+12+CDR+Specification

> Memory leak during perf testing
> -------------------------------
>
>                 Key: ASTERISK-25905
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-25905
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Applications/app_confbridge, pjproject/pjsip
>    Affects Versions: 13.8.0
>         Environment: Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Linux ykt1cfbprd1 3.10.0-327.13.1.el7.x86_64
> certified/13.8-cert1-rc1
> pjproject-2.4.5
>            Reporter: Robert McGilvray
>            Assignee: Richard Mudgett
>            Severity: Minor
>         Attachments: loadtest.txt, memory-summary.txt
>
>
> ** I've been testing against the certified branch, last cloned yesterday with certified/13.8-cert1-rc1. It would not allow me to select that as a version however ** 
> While using sipp as a generator to load test Asterisk I've come across a memory leak that very quickly exhausts the host of resources. 
> The testing methodology is pretty simple: use sipp to launch 1500 concurrent calls to asterisk with a call rate of 25/sec. On the asterisk side use the RAND function to generate two numbers, one of which is the confbridge number and the other (either 0 or 1) is to determine whether to use the moderator profile or participant. The call is then dropped into a ConfBridge for 60s and Hungup. 
> After a few thousand completed calls the memory usage grows and eventually exhausts the host resources. I recompiled with MALLOC_DEBUG enabled, the output of memory show allocations is attached. It looks like the allocations are in stasis_channels, well after all channels have been disconnected. 
> {noformat}
> ykt1cfbprd1:/home/netops# ps -C asterisk u
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> asterisk 20927  113  9.2 6292308 3029204 ?     Sl   16:32  24:47 /home/asterisk/asterisk-cert-13.8/sbin/asterisk -f -C /home/asterisk/asterisk-cert
> root     32052  0.0  0.0  47428  2840 pts/0    S+   16:43   0:00 rasterisk risk/asterisk-cert-13.8/sbin/asterisk -r
> {noformat}
> Please let me know if you need any further information.
> Thanks!!



--
This message was sent by Atlassian JIRA
(v6.2#6252)