<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/2426/">https://reviewboard.asterisk.org/r/2426/</a>
</td>
</tr>
</table>
<br />
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Testing before and after proposed fix, which will be to check the iostate == IDLE before sleeping:
in iax2_process_thread()
with;
} else {
ast_log(LOG_WARNING, "ALEC[%d] SLEEP state[%d] actions[%d]\n", thread->threadnum, thread->iostate, thread->actions);
ast_cond_wait(&thread->cond, &thread->lock);
ast_log(LOG_WARNING, "ALEC[%d] AWAKE state[%d] actions[%d]\n", thread->threadnum, thread->iostate, thread->actions);
}
just after module load chan_iax2.so
[Apr 4 22:06:29] WARNING[26229]: chan_iax2.c:11915 iax2_process_thread: ALEC[4] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26230]: chan_iax2.c:11915 iax2_process_thread: ALEC[5] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26231]: chan_iax2.c:11915 iax2_process_thread: ALEC[6] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26232]: chan_iax2.c:11915 iax2_process_thread: ALEC[7] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26233]: chan_iax2.c:11915 iax2_process_thread: ALEC[8] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26234]: chan_iax2.c:11915 iax2_process_thread: ALEC[9] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26235]: chan_iax2.c:11915 iax2_process_thread: ALEC[10] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26236]: chan_iax2.c:9768 socket_read: ALEC[1] send on it's way state[1]
[Apr 4 22:06:29] WARNING[26228]: chan_iax2.c:11915 iax2_process_thread: ALEC[3] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26227]: chan_iax2.c:11915 iax2_process_thread: ALEC[2] SLEEP state[0] actions[0]
[Apr 4 22:06:29] WARNING[26226]: chan_iax2.c:11915 iax2_process_thread: ALEC[1] SLEEP state[1] actions[0] << shouldn't be asleep!!!!
with;
} else if (thread->iostate == IAX_IOSTATE_IDLE){
ast_log(LOG_WARNING, "ALEC[%d] SLEEP state[%d] actions[%d]\n", thread->threadnum, thread->iostate, thread->actions);
ast_cond_wait(&thread->cond, &thread->lock);
ast_log(LOG_WARNING, "ALEC[%d] AWAKE state[%d] actions[%d]\n", thread->threadnum, thread->iostate, thread->actions);
}
just after module load chan_iax2.so
[Apr 4 22:13:50] WARNING[29135]: chan_iax2.c:11915 iax2_process_thread: ALEC[4] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29136]: chan_iax2.c:11915 iax2_process_thread: ALEC[5] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29137]: chan_iax2.c:11915 iax2_process_thread: ALEC[6] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29138]: chan_iax2.c:11915 iax2_process_thread: ALEC[7] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29139]: chan_iax2.c:11915 iax2_process_thread: ALEC[8] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29140]: chan_iax2.c:11915 iax2_process_thread: ALEC[9] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29141]: chan_iax2.c:11915 iax2_process_thread: ALEC[10] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29142]: chan_iax2.c:9768 socket_read: ALEC[1] send on it's way state[1]
[Apr 4 22:13:50] WARNING[29134]: chan_iax2.c:11915 iax2_process_thread: ALEC[3] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29133]: chan_iax2.c:11915 iax2_process_thread: ALEC[2] SLEEP state[0] actions[0]
[Apr 4 22:13:50] WARNING[29132]: chan_iax2.c:11942 iax2_process_thread: ALEC[1] processing state[2] actions[1] <<< correct
[Apr 4 22:13:50] WARNING[29132]: chan_iax2.c:11915 iax2_process_thread: ALEC[1] SLEEP state[0] actions[1] <<< correct
</pre>
<br />
<p>- Alec</p>
<br />
<p>On April 4th, 2013, 7:23 a.m. UTC, Alec Davis wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/static/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers.</div>
<div>By Alec Davis.</div>
<p style="color: grey;"><i>Updated April 4, 2013, 7:23 a.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="https://issues.asterisk.org/jira/browse/ASTERISK-18827">ASTERISK-18827</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
Asterisk
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">On a slower machine when a full_frame with the same callno is already being processed by another thread the frame is queued onto the other thread.
However, nothing indicates to the other thread that it now has a frame to process.
On the bug report, I have debug messages in the patch.
In theory, the thread that is already processing the callno should pickup the deferred frames, but it doesn't.
What was also revealed in ASTERISK-18827 using iax_18827.diff2.txt, is that the deferred frames queue increases each time a POKE/PONG is initiated.
At some stage this will exhaust memory.
Refer to the line below
+ ast_log(LOG_WARNING, "ALEC[%d] unhandled frame on thread [%d]\n", from_here->threadnum, to_here->threadnum);
</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Before patch:
astrid-test*CLI> iax2 show threads
IAX2 Thread Information
Idle Threads:
Thread 2: state=0, update=5, actions=4, func=''
Thread 3: state=0, update=4, actions=2, func=''
Thread 4: state=0, update=10, actions=2, func=''
Thread 5: state=0, update=3, actions=4, func=''
Thread 6: state=0, update=8, actions=2, func=''
Thread 7: state=0, update=2, actions=3, func=''
Thread 8: state=0, update=1, actions=3, func=''
Thread 9: state=0, update=0, actions=3, func=''
Thread 10: state=0, update=6, actions=1, func=''
Active Threads:
Thread P1: state=1, update=1365045454, actions=0, func='socket_process'
Dynamic Threads:
10 of 10 threads accounted for with 0 dynamic threads
and iax2 show peers reports link is UNREACHABLE
After patch:
iax2 show threads has all idle.
iax2 show peers (immediately after 'module load chan_iax2.so')
Name/Username Host Mask Port Status Description
astrid2/astrid2 192.168.xxx.yyy (S) 255.255.255.255 4569 (T) OK (1002 ms)
1 iax2 peers [1 online, 0 offline, 0 unmonitored]
</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>trunk/channels/chan_iax2.c <span style="color: grey">(376441)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/2426/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>