[asterisk-bugs] [JIRA] (ASTERISK-28962) Asterisk Memory Leak after SIP Reply Flood

[Jonas Swiatek (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=251219#comment-251219 ] 

Jonas Swiatek edited comment on ASTERISK-28962 at 6/24/20 3:25 PM:
-------------------------------------------------------------------

The logs literally contains nothing else than the "WARNING[23382] channel.c: Exceptionally long queue length queuing to PJSIP/registrar-0000ab85" entries

The only thing prior to it in /var/log/asterisk/messages is a few hours before:
NOTICE[8467][C-00004431] app_dial.c: Not accepting call completion offers from call-forward recipient Local/0544445 at dial-00003ee2;1

I'll see if I can crank up the log verbosity on the server. It's only serving that specific customer right now because I don't want his crazy SIP client to nuke my server again ;)

Endpoint config:
{code}
[registrar]
type=identify
endpoint=registrar
match=<redacted>
match=<redacted>

[registrar]
type=aor
contact=<redacted>

[registrar]
type=endpoint
transport=transport-udp
context=dial
disallow=all
allow=ulaw,alaw,opus
aors=registrar
force_rport=yes
direct_media=no
rtp_symmetric=yes
refer_blind_progress=no
media_address=<redacted>
media_encryption=sdes
media_encryption_optimistic=yes
rtp_timeout=120
{code}

Dialplan:
{code}
[dial]
exten => _.,1,NoOP(AGI: ${agiUrl})
same => n,GosubIf($[${LEN(${agiUrl})} = 0]?init-agi,SetAgi,1)
same => n,AGI(agi://${agiUrl}/dial)
same => n,Hangup()
{code}
We use Fast AGI for everything, so the call would've been handled by that at the time it crashed out. Maybe that's a relevant factor?



was (Author: jonasswiatek):
The logs literally contains nothing else than the "WARNING[23382] channel.c: Exceptionally long queue length queuing to PJSIP/registrar-0000ab85" entries

The only thing prior to it in /var/log/asterisk/messages is a few hours before:
NOTICE[8467][C-00004431] app_dial.c: Not accepting call completion offers from call-forward recipient Local/0544445 at dial-00003ee2;1

I'll see if I can crank up the log verbosity on the server. It's only serving that specific customer right now because I don't want his crazy SIP client to nuke my server again ;)

Endpoint config:
[registrar]
type=identify
endpoint=registrar
match=<redacted>
match=<redacted>

[registrar]
type=aor
contact=<redacted>

[registrar]
type=endpoint
transport=transport-udp
context=dial
disallow=all
allow=ulaw,alaw,opus
aors=registrar
force_rport=yes
direct_media=no
rtp_symmetric=yes
refer_blind_progress=no
media_address=<redacted>
media_encryption=sdes
media_encryption_optimistic=yes
rtp_timeout=120

Dialplan:
[dial]
exten => _.,1,NoOP(AGI: ${agiUrl})
same => n,GosubIf($[${LEN(${agiUrl})} = 0]?init-agi,SetAgi,1)
same => n,AGI(agi://${agiUrl}/dial)
same => n,Hangup()

We use Fast AGI for everything, so the call would've been handled by that at the time it crashed out. Maybe that's a relevant factor?


> Asterisk Memory Leak after SIP Reply Flood
> ------------------------------------------
>
>                 Key: ASTERISK-28962
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28962
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Channels/chan_pjsip
>    Affects Versions: 16.6.1
>         Environment: Amazon Linux 2 (CentOS based)
>            Reporter: Jonas Swiatek
>            Assignee: Unassigned
>
> I've got a user with a SIP Client (MicroSIP) that some times goes haywire, and responds to the initial INVITE from Asterisk by retransmitting it's 180 Ringing on a loop, seemingly unbounded. We're talking thousands of the same reply. I haven't been able to retrieve all of them from Homer as there are simply too many for their UI to cooperate. But the 100 or so I did manage to pull was all sent within a handful of milliseconds.
> When Asterisk received these, it's memory usage shots up significantly, and the log file started printing hundreds of these lines:
> WARNING[13337] channel.c: Exceptionally long queue length queuing to PJSIP/registrar-0000d96a
> The channel noted is the outgoing call to the user with the defective SIP Client.
> There was nothing else in the messages log file related to this at all.
> Asterisk CPU spiked (thought not to 100%), which is not a surprise as it does need to process all these SIP Replies, however the memory usage did not go down afterwards, even 12 hours after.
> Normally it'll sit around 5 mb or something, but after this it had allocated a good 6GB of ram and there was no sign it was about to give any of it up 12 hours later when I restarted it.
> I've got a PCAP file, though it contains sensitive information so I'd prefer not to share it publicly
> I've also moved this customer to a separate Asterisk instance running 16.11.1, and I can probably enable whatever other logging might be helpful on it, in case this client goes haywire again and takes out that instance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)