[asterisk-dev] scalability issue with realtime lastms update on qualify state change

[Damon Estep]

> On Sat, Nov 16, 2013 at 2:02 AM, Olle E. Johansson <oej at edvina.net> wrote:
> > Yes, there's one single thread in chan_sip handling all the Qualify
> > and inbound sip messages. If we moved the database updates to a
> > background thread the situation would improve. Then the monitor thread
> > would just move on to the next message, letting another thread wait for the
> database.
> 
> Secondarily, the pokes (qualifies) are sent in each peer thread.  If instead, a
> single background thread were sending the qualifies, (possibly even rate limiting
> the sending, in order to ensure the responses can be received on time), that
> would additionally eliminate the flapping.  The issue is that the return packets
> cannot be acted upon before the expiration time, because the initial poke
> packets are sent in parallel, but the responses have to be received serially, within
> a single thread.
> 
> I disagree with the alternate coding proposal, because it attempts to code
> around the problem by turning something off, rather than attacking the core of
> the problem, which solves it for everybody.
> 
> -Tilghman
> 
Well, out of desperation I have written a patch that allows the user to disable realtime updates of lastms with a configuration setting of rtlastms=no. This at least lets me turn off database updates on qualify state change until there is a better solution, and I agree that the solution needs to be a better one. I will be testing in a lab and then in production over the next week.

Even with a separate thread for qualify I would still argue that lastms realtime database updates should be configurable.

rtupdate was introduced to toggle registration/expiration updates to the realtime database. Sometime after rtupdate was introduced qualify state changes were added to the list of realtime database updates. Qualify state changes are not registration information, so at that time it would have been a good idea to make them optional, since the amount of new database traffic can be substantial on heavily loaded systems.

In summary, I believe being able to toggle lastms realtime updates on and off independently of realtime registration updates makes a lot of sense, and provides an immediate solution to a real problem, while not impacting users that rely on it, particularly if the default setting is on.

In terms of the better solution, I am not familiar enough with all of the code and not competent enough to tackle the project of moving the qualify and subsequent database updates to a separate thread. My company on the other hand might be willing to contribute a few $ to the cause if there is someone with the time and skills to tackle it.  Any idea what kind of project that would be in terms of time? 

Damon