<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/2441/">https://reviewboard.asterisk.org/r/2441/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On April 15th, 2013, 5:37 p.m. UTC, <b>Matt Jordan</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I'm not sure I can fully articulate what is happening here yet, so bear with me.
Assume we are using res_timing_pthread and this patch. When a silence generator is on a channel and the channel timer is put into continuous mode (which occurs during play_and_record), there appears to be a race condition between the timer being serviced - where write_byte is called and pending_ticks is incremented - and the thread servicing the timer acknowledging the timer properly. At some point, timer->pending_ticks is incremented past 1, at which point all hell breaks loose.
By "all hell breaking loose", the channel in question which has the audio streamed on it is no longer serviced; the dreaded "exceptionally long queue" message is displayed; the channel cannot be hung up; and the CPU pegs out. This so far can only be reproduced if the silence generator on record option is enabled in asterisk.conf and a Local channel is put into VoiceMail.
A simple fix thus far has been to put the following at the beginning of write_byte:
if (timer->pending_ticks) {
return;
}
timer->pending_ticks++;
This means we can only be in overflow (which would be a very strange situation to get into, and would only happen if someone in a future patch incremented pending_ticks elsewhere) or enabled.
Although this patch did prevent the situation I was seeing, I did notice that the silence generator will still sometimes prevent Asterisk from detecting silence on a channel and killing the recording. That's potentially a separate bug (and does not happen every time).</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">As a follow up: I was running an old version of the patch, and the most recent version of the patch on the issue doesn't exhibit any of the problems I was seeing.</pre>
<br />
<p>- Matt</p>
<br />
<p>On April 11th, 2013, 10:16 p.m. UTC, Shaun Ruffell wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/static/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers and Matt Jordan.</div>
<div>By Shaun Ruffell.</div>
<p style="color: grey;"><i>Updated April 11, 2013, 10:16 p.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="https://issues.asterisk.org/jira/browse/ASTERISK-21389">ASTERISK-21389</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
Asterisk
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">From the patch attached on ASTERISK-21389:
res_timing_pthread: Reduce probability of deadlocking on timer.
There were several reports of deadlock when using
res_timing_pthread. Backtraces indicated that one thread was blocked
waiting for the write to the pipe to complete and this thread held
the container lock for the timers. Therefore any thread that wanted
to create a new timer or read an existing timer would block waiting
for either the timer lock or the container lock and deadlock ensued.
This patch changes the way the pipe is used to eliminate this source
of deadlocks:
1) The pipe is placed in non-blocking mode so that it would never
block even if the following changes someone fail...
2) Instead of writing bytes into the pipe for each "tick" that's
fired the pipe now has two states--signaled and unsignaled. If
signaled, the pipe is hot and any pollers of the read side
filedescriptor will be woken up. If unsigned the pipe is idle. This
eliminates even the chance of filling up the pipe and reduces the
potential overhead of calling unnecessary writes.
3) Since we're tracking the signaled / unsignaled state, we can
eliminate the exta poll system call for every firing because we know
that there is data to be read.
</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">ran timing test on the console and also put several channels in a confbridge.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/trunk/res/res_timing_pthread.c <span style="color: grey">(385382)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/2441/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>