[asterisk-dev] [Code Review] Improve frame_queue handing in chan_iax2

David Vossel dvossel at digium.com
Tue Mar 31 10:42:38 CDT 2009


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://reviewboard.digium.com/r/212/#review653
-----------------------------------------------------------

Ship it!


Rockin!

- David


On 2009-03-29 00:04:37, Russell Bryant wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://reviewboard.digium.com/r/212/
> -----------------------------------------------------------
> 
> (Updated 2009-03-29 00:04:37)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Summary
> -------
> 
> While doing some profiling for <http://reviewboard.digium.com/r/205/>, one function that caught my eye was network_thread() in chan_iax2.c.  After the things that I was working on there, it was the next target for analysis and optimization.  I used oprofile's source annotation functionality and found that the loop traversing the frame queue in network_thread() was to blame for the excessive CPU cycle consumption.
> 
> The frame_queue in chan_iax2 previously held all frames that either were pending transmission or had been transmitted and are still pending acknowledgment.
> 
> In network_thread(), the previous code would go back through the main for loop after reading a single incoming frame or after being signaled because a frame had been queued up for initial transmission.  In each iteration of the loop, it traverses the entire frame queue looking for frames that need to be transmitted.  On a busy server, this could easily be quite a few entries.
> 
> This patch is actually quite simple.  The frame_queue has become only a list of frames pending acknowledgment.  Frames that need to be transmitted are queued up to a dedicated transmit thread via the taskprocessor API.
> 
> As a result, the code in network_thread() becomes much simpler, as its only job is to read incoming frames.
> 
> ---------
> 
> In addition to the previously described changes, this patch now includes some additional changes to the frame_queue.  Instead of one big frame_queue, now there is a list per call number to further reduce wasted list traversals.  The biggest impact of this change is in socket_process().
> 
> 
> Diffs
> -----
> 
>   /trunk/channels/chan_iax2.c 184835 
> 
> Diff: http://reviewboard.digium.com/r/212/diff
> 
> 
> Testing
> -------
> 
> I used a similar test that I used in R205 for this.  I sent 15000 IAX2 registrations evenly spaced out over 1 minute.  The server was configured using realtime with a MySQL backend via res_config_odbc.  IAX2 realtime caching was enabled.  I ran the test, used oprofile, and watched CPU usage.
> 
> The box has an Intel Core 2 Duo @ 2.33 GHz and 2 GB of RAM.  The noted CPU consumption is out of 200%, since it's a dual core machine.
> 
> ----------------------------
> 
> Asterisk SVN-trunk-r184801 built by russell @ milo on a x86_64 running Linux on 2009-02-12 19:26:12 UTC
> 
> Average CPU usage by Asterisk: 10% to 15%
> 
> (results below 0.5% omitted)
> 
> samples  %        image name               app name                 symbol name
> 33690    39.8500  chan_iax2.so             chan_iax2.so             network_thread
> 13263    15.6881  chan_iax2.so             chan_iax2.so             socket_process
> 6972      8.2468  chan_iax2.so             chan_iax2.so             __attempt_transmit
> 6618      7.8281  chan_iax2.so             chan_iax2.so             pvt_destructor
> 6542      7.7382  chan_iax2.so             chan_iax2.so             __find_callno
> 2037      2.4095  asterisk                 asterisk                 internal_ao2_callback
> 620       0.7334  asterisk                 asterisk                 sched_time_cmp
> 602       0.7121  asterisk                 asterisk                 handle_statechange
> 563       0.6659  [vdso] (tgid:7462 range:0x7fffe31fe000-0x7fffe3200000) asterisk                 (no symbols)
> 555       0.6565  chan_iax2.so             chan_iax2.so             peer_cmp_cb
> 518       0.6127  asterisk                 asterisk                 internal_ao2_ref
> ...
> 
> ----------------------------
> 
> (With only first set of changes)
> 
> Asterisk SVN-russell-iax2_transmit_q-r184836-/trunk built by root @ milo on a x86_64 running Linux on 2009-03-29 03:43:13 UTC
> 
> Average CPU usage by Asterisk: 5% to 10%
> 
> (most results below 0.5% omitted)
> 
> samples  %        image name               app name                 symbol name
> 9290     24.5969  chan_iax2.so             chan_iax2.so             socket_process
> 6553     17.3502  chan_iax2.so             chan_iax2.so             __find_callno
> 3630      9.6111  chan_iax2.so             chan_iax2.so             pvt_destructor
> 2392      6.3332  chan_iax2.so             chan_iax2.so             __attempt_transmit
> 1946      5.1524  asterisk                 asterisk                 internal_ao2_callback
> 582       1.5409  asterisk                 asterisk                 handle_statechange
> 557       1.4748  chan_iax2.so             chan_iax2.so             peer_cmp_cb
> 528       1.3980  asterisk                 asterisk                 internal_ao2_ref
> 523       1.3847  [vdso] (tgid:3088 range:0x7fff3c3fe000-0x7fff3c400000) asterisk                 (no symbols)
> 478       1.2656  asterisk                 asterisk                 sched_time_cmp
> 416       1.1014  res_config_odbc.so       res_config_odbc.so       realtime_odbc
> 348       0.9214  chan_iax2.so             chan_iax2.so             register_verify
> 339       0.8976  chan_iax2.so             chan_iax2.so             build_peer
> 320       0.8473  chan_iax2.so             chan_iax2.so             .plt
> 291       0.7705  asterisk                 asterisk                 __bt_defcmp
> 236       0.6249  asterisk                 asterisk                 _ast_heap_remove
> 233       0.6169  asterisk                 asterisk                 .plt
> 221       0.5851  asterisk                 asterisk                 __ao2_iterator_next
> 216       0.5719  asterisk                 asterisk                 tps_processing_function
> 204       0.5401  asterisk                 asterisk                 ast_get_channel_tech
> 193       0.5110  chan_iax2.so             chan_iax2.so             iax2_process_thread
> 190       0.5031  asterisk                 asterisk                 ast_hashtab_remove_this_object
> ...
> 56        0.1483  chan_iax2.so             chan_iax2.so             transmit_frame
> ...
> 36        0.0953  chan_iax2.so             chan_iax2.so             network_thread
> ...
> 
> ----------------------------
> 
> (Now, with the frame_queue changed to an array of lists.)
> 
> Asterisk SVN-russell-iax2_transmit_q-r184837-/trunk built by root @ milo on a x86_64 running Linux on 2009-03-29 03:43:13 UTC
> 
> Average CPU usage by Asterisk: 3% to 6%
> 
> (results below 1% omitted)
> 
> samples  %        image name               app name                 symbol name
> 6413     27.6816  chan_iax2.so             chan_iax2.so             __find_callno
> 1929      8.3265  asterisk                 asterisk                 internal_ao2_callback
> 707       3.0518  chan_iax2.so             chan_iax2.so             socket_process
> 558       2.4086  asterisk                 asterisk                 internal_ao2_ref
> 550       2.3741  asterisk                 asterisk                 sched_time_cmp
> 524       2.2618  asterisk                 asterisk                 handle_statechange
> 522       2.2532  [vdso] (tgid:5835 range:0x7fff071fe000-0x7fff07200000) asterisk                 (no symbols)
> 516       2.2273  chan_iax2.so             chan_iax2.so             peer_cmp_cb
> 368       1.5885  res_config_odbc.so       res_config_odbc.so       realtime_odbc
> 319       1.3770  chan_iax2.so             chan_iax2.so             register_verify
> 310       1.3381  chan_iax2.so             chan_iax2.so             .plt
> 302       1.3036  chan_iax2.so             chan_iax2.so             build_peer
> 288       1.2431  asterisk                 asterisk                 _ast_heap_remove
> 273       1.1784  asterisk                 asterisk                 __bt_defcmp
> 268       1.1568  asterisk                 asterisk                 .plt
> 250       1.0791  asterisk                 asterisk                 __ao2_iterator_next
> ...
> 
> 
> Thanks,
> 
> Russell
> 
>




More information about the asterisk-dev mailing list