<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/1535/">https://reviewboard.asterisk.org/r/1535/</a>
</td>
</tr>
</table>
<br />
<p>Ship it!</p>
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/1535/diff/1/?file=21268#file21268line3084" style="color: black; font-weight: bold; text-decoration: underline;">/branches/10/apps/app_queue.c</a>
<span style="font-weight: normal;">
(Diff revision 1)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">static int ring_entry(struct queue_ent *qe, struct callattempt *tmp, int *busies)</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">3084</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">newstate</span> <span class="o">=</span> <span class="n">ast_device_state</span><span class="p">(</span><span class="n">tmp</span><span class="o">-></span><span class="n">member</span><span class="o">-></span><span class="n">interface</span><span class="p">);</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This may still involve a walk of the channel list (and it my testing it did involve a few), but it performs much better than calling ast_device_state_parse() directly.</pre>
</div>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/1535/diff/1/?file=21268#file21268line3086" style="color: black; font-weight: bold; text-decoration: underline;">/branches/10/apps/app_queue.c</a>
<span style="font-weight: normal;">
(Diff revision 1)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">static int ring_entry(struct queue_ent *qe, struct callattempt *tmp, int *busies)</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">3086</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">ast_log</span><span class="p">(</span><span class="n">LOG_ERROR</span><span class="p">,</span> <span class="s">"Found a channel matching iterface %s while status was %i changed to %i</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Use ast_devstate2str here to format the device state names.</pre>
</div>
<br />
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">There appears to be some sort of race condition between when a device state is updated and when we try to ring a device. I don't understand how all this works well enough to know for sure, but even with this change it still seems possible that the device state could change between when we check here and when we actualy dial. It is impossible to get this 100% perfect, as we can't control who may be dialing the device outside of asterisk, but I suppose we could make this work right within asterisk. Either way, that is outside of the limited scope of this change.
This change looks good. In my performance testing it has a much much lesser effect on performance than the previous code that was here. There are still some channel list walks involved, but they don't appear to be causing massive performance degradation like the previous change did.</pre>
<p>- Matthew</p>
<br />
<p>On October 20th, 2011, 5:49 a.m., irroot wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers and Matthew Nicholson.</div>
<div>By irroot.</div>
<p style="color: grey;"><i>Updated Oct. 20, 2011, 5:49 a.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">commit r325483 introduced a performance problem reverted by r341486
here is a proposal to do this more "friendly" its a global option [possibly should be
per Q] and uses ast_device_state that tries to get the state from a channel driver
it calls ast_parse_device_state only if this fails to yield a result it then fires a
devicestate event to set it so it does not get called over and over.
the reality is its happening 4 out of 1000 times this check is run that the agent is busy
this is a minimal % the customer who raised this issue is under strict SLA to answer calls
and provide service they a medical aid call center who deal with e911 cases authorizing
treatment.
hope this rework is more acceptable ill be putting this in production and reporting results back.
in the log files [rotated daily] from 15/09/2011 to 19/10/2011
110507 times was logged
516 duplicate calls prevented
ill be watching these metrics closely in next bit.
the problem is its not efficient at all as they only had 46598 calls come into the system.
-------------
The regression was caused by a call to ast_parse_device_state() in app_queue's
ring_entry() function. The ast_parse_device_state() function eventually calls
ast_channel_get_full() with a channel name prefix which causes it to walk the
channel list causing massive lock contention and slow downs.
This patch fixes the regression by removing the call to
ast_parase_device_state() which should be unnecessary. Queue member device
state should be maintained by device state events. Some users have seen
instances where busy agents were called when they shouldn't have, which is the
reason the call to ast_parse_device_state() was added. That change appears to
have resolved that issue but also causes this performance regression. There may
still be issues with queue member status, and if so, alternative methods should
be investigated to resolve them.</pre>
</td>
</tr>
</table>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="https://issues.asterisk.org/jira/browse/AST-695">AST-695</a>
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/branches/10/apps/app_queue.c <span style="color: grey">(341525)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/1535/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>