[asterisk-users] 13.22.0 - HTTP session count exceeded 100 sessions - instance unusable

[Stefan Viljoen]

Hi Joshua

No back-off, but I am caching the last 5000 result and and first hitting the cache to see if a recent command already provided the information I'm seeking for a particular request.




From: Joshua C. Colp <jcolp at sangoma.com> 
Sent: Wednesday, 01 July 2020 15:47
To: viljoens at verishare.co.za; Asterisk Users Mailing List - Non-Commercial Discussion <asterisk-users at lists.digium.com>
Subject: Re: [asterisk-users] 13.22.0 - HTTP session count exceeded 100 sessions - instance unusable

On Wed, Jul 1, 2020 at 10:32 AM Stefan Viljoen <mailto:viljoens at verishare.co.za> wrote:
Hi Joshua

HTTP is used on in our setup on

http://127.0.0.1/mxml?<command>

to send commands to the server, such as

http://127.0.0.1/mxml?action=login&username=myuser&secret=thesecret

to log in and then

http://127.0.0.1/mxml?ActionID=123&Action=BlindTransfer&Channel=Channel&Context=local&Exten=123&Priority=1

etc. to control transfers, for example.

ARI is not being used.

WebRTC is not being used.

I have in the meantime modified httpd.conf and halved the session_inactivity from 30 000 to 20 000 and session_keep_alive from 15 000 to 10 000, and increased the sessionlimit to 1000 (instead of 100) to see if shorter sessions expiring more frequently and just plain more available slots for sessions will help?

So far so good, done about 55 000 new calls that way the past 32 minutes.

Or am I barking up the wrong tree?

It's possible it will help - but that still doesn't explain what exactly happened. Does your code have a back off strategy if HTTP requests fail? If not and they have an immediate retry with loop, then that may have effectively DDoSed things. It would also be good to explain why 100 connections were in use - did old connections not get closed? Was your load sufficient to just meet that amount? 

-- 
Joshua C. Colp
Asterisk Technical Lead
Sangoma Technologies
Check us out at http://www.sangoma.com and http://www.asterisk.org