[asterisk-bugs] [JIRA] (ASTERISK-24291) res_srtp module stops working after about 35.000 processed calls

Thu Sep 11 17:00:29 CDT 2014

    [ https://issues.asterisk.org/jira/browse/ASTERISK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=222626#comment-222626 ] 

Robert H. commented on ASTERISK-24291:
--------------------------------------

Thanks for your feedback Rusty.

This is a production system with heavy load, but I can switch main traffic to another prod machine tomorrow to reproduce the issue with the loadtest-script.
As I know that it happens every 34-35k calls I can enable rtp debug some few hundred calls before the issue occurs.
The test scenario simulates nearly the same behavior which we have in production (busy calls, answered calls with different call lengths, cancelled calls an so on).

For load testing and to reproduce the issue I use 2 additional Asterisk machines that behave like the clients behind the Asterisk Routing Servers:

Server 1: Dial Out Asterisk Test Server (calls Asterisk Server 2 with unencrypted RTP traffic)
Server 2+3: Asterisk Production Servers. They route traffic between Server 2+3 over WAN with SIP-TLS and SRTP. The issue happens on both servers.
Server 4: Asterisk Test call receiver (gets calls from Server 3 unencrypted)

Asterisk 1+2 and 3+4 are on separate physical networks. Link between server 2+3 is a dedicated symmetric 100MBit WAN Link.
Servers 2 has two network cards with different vlans for internal/external traffic.
Server 3 has one network card with different vlans for internal/external traffic.

sip.conf on Server 2 as well as Server 3 is quite simple:
{noformat}
;--- start sip.conf
[general]
context=default         
match_auth_username=yes 
allowoverlap=yes        
udpbindaddr=0.0.0.0                 
nat=auto_force_rport,auto_comedia
tcpenable=no            
tcpbindaddr=0.0.0.0     
tlsenable=yes
tlsbindaddr=0.0.0.0
tlscertfile=/etc/asterisk/keys/asterisk.pem
tlscafile=/etc/asterisk/keys/ca.crt
tlscipher=ALL
tlsclientmethod=tlsv1 
tlsdontverifyserver=yes
transport=udp      
srvlookup=yes      
tos_sip=cs3            
tos_audio=ef           
tos_video=af41         
tos_text=af41          
cos_sip=3              
cos_audio=5            
cos_video=4            
cos_text=3             
useragent=Asterisk PBX
legacy_useroption_parsing=yes
use_q850_reason = yes
session-timers=refuse
sdpsession=Asterisk PBX

[authentication]

[tpl_ivr](!)
context=trunk_ivrs
type=peer
dtmfmode=rfc2833
qualify=no
port=5060
usereqphone=yes	
callcounter=yes
disallow=all
allow=alaw
allow=ulaw
sendrpid = no
trustrpid = no
disallowed_methods = UPDATE
accountcode=ivr_not_set
directmedia=no
deny=0.0.0.0/0.0.0.0 
permit=10.100.1.0/27

[ivr3vlan54](tpl_ivr)
host=10.100.1.23	; internal dial out IVR on vlan54

[bccc]
context=trunk_bccc
secret=none		; not used. we have insecure=invite with an acl
type=friend  
dtmfmode=info
host=X.X.X.X	; WAN IP of Asterisk 3 Server
directmedia=no
nat=no
qualify=yes
disallow=all
allow=alaw         
allow=ulaw         
accountcode=bccc
callcounter=yes
deny=0.0.0.0/0.0.0.0
permit=x.x.x.x/32	 ; WAN IP of Asterisk 3 Server
transport=tls
encryption=yes		; CAUTION! when enabled for SRTP, jira issue 24291 occurs 
progressinband=yes
tonezone=de
insecure=invite

;--- end of sip.conf
{noformat}

I will reply tomorrow with traces when I have finished load tests with rtp debugging enabled.
In addition, I will send you the involved extensions.conf parts to reproduce it.

Thank you,
Robert

> res_srtp module stops working after about 35.000 processed calls
> ----------------------------------------------------------------
>
>                 Key: ASTERISK-24291
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-24291
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Channels/chan_sip/SRTP, Resources/res_srtp
>    Affects Versions: 11.11.0, 11.12.0
>         Environment: Ubuntu 12.04.5 LTS (GNU/Linux 3.13.0-34-generic x86_64) running on HP DL360 G6/7, latest  libsrtp0 version 1.4.4+20100615~dfsg-1build, SIP only environment
>            Reporter: Robert H.
>            Assignee: Robert H.
>            Severity: Critical
>         Attachments: issue_24291_full_log.14.txt
>
>
> When using encryption for RTP streams, asterisk does not accept any calls after about 35k calls (reproducable) have been processed correctly.
> All further inbound and outbound calls are rejected with a 
> "488 - Not Acceptable Here".
> When this happens, one asterisk machine shows:
> {noformat}
> [2014-08-29 17:32:23.807] DEBUG[28500][C-00009387]: chan_sip.c:10530 process_sdp: Processing media-level (audio) SDP a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:cYswzW2zYpdgsVkKgQWvdbUSLedzlE8nByMqEYiI... UNSUPPORTED OR FAILED.
> [2014-08-29 17:32:23.807] WARNING[28500][C-00009387]: chan_sip.c:10535 process_sdp: Rejecting secure audio stream without encryption details: audio 11070 RTP/SAVP 8 0 101
> {noformat}
> the destination asterisk shows:
> {noformat}
> WARNING[10222][C-0000883a]: chan_sip.c:12925 get_crypto_attrib: No SRTP key management enabled
> {noformat}
> Active srtp calls are not affected when this issue occurs, but all further Invites are rejected with the 488 response, so no more calls can be processed.
> The only solution at the moment is to restart asterisk or to wait until no more SRTP calls are active and then unload res_srtp.so followed by loading the module again.
> add info:
> - problem occurs regardless of using SIP over TLS or SIP without TLS
> - other (unencrypted) RTP connections are still working  
> If you need further info, just let me know.
> Thanks for checking into this!
> Robert
>     

--
This message was sent by Atlassian JIRA
(v6.2#6252)