[asterisk-dev] chan_sip -- improving the speed of sip reloads

[Luigi Rizzo]

On Fri, Nov 16, 2007 at 09:21:46AM -0600, Steve Murphy wrote:
> Sip officianados--- 
> 
> Olle closed 11210, where the reporter has 5000 entries in his sip.conf file, and network processing is put on hold for over 40 seconds while a reload is being done, which causes some fairly significant problems. 
> 
> This looked like a fairly juicy problem, so I started work on it, but about 1/3 of the way thru it, Olle closed that bug, preferring that the issue be taken to asterisk-dev as an architectural issue instead. 
> 
> So, here I am. Here is what I've done, and what I'm planning to do; let me know if I'm going in the wrong direction. 

lots of issues below so let me just address a small set of them

> work will be involved in those things that astobj provided, that astobj2 does not. (marking, etc) 

i don't think marking is a problem at all - astobj2 gives you the
chance to run a function on all objects so the function could just
do the filtering and marking as appropriate (in the object, outside,
it is really up to you and how you write the function). There is also
support for doing more sophisticated things such as deleting matching
objects and so on.

> Since astobj2 hash tables are not automatically resizeable ...,

let's move by small steps. Moving from a single list to a fixed size
table (maybe configurable at runtime) is already a significant
improvement and we should be already happy with that.

Then we could work on adding support for resizing to astobj2 and
that should give us the extra performance we need.

for chan_sip, i think we had the dialog list converted to astobj2
in the team/rizzo/astobj2 branch.

> I noticed that chan_iax2 has several traversals of hash tables, but those in iax2_getpeername and iax2_getpeertrunk look as though a new hashtable, based on sin_addr and sin_port fields (ip addresses/ports) as key could speed up the searches. I don't know if these functions are called often or not... if they are, it might be worth the work to form, and maintain this additional hash table. The same situtation seems to exist in chan_sip, where traversals of peers (at least) are done, looking for a match to sin_addr and sin_port numbers. Such hash-table traversals in astobj2 make me nervous.
>...  Some time back, I reported that a 1% mix of traversals in a multi-threaded environment slowed astobj2 down by 15x. In other words, a benchmark that normally took 60 seconds to run, took 15 minutes to run when traversals of the entire hash table were done once every 100 operations. This behavior hasn't been corrected yet, as far as I know.

well if i remember well, you used iterators for those measurements,
which means a lock contention on every single step. For those kind of
operations the correct way would be to use the callback mechanism,
which iterates over the container without contending for the lock
at each step.

cheers
luigi