<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/1430/">https://reviewboard.asterisk.org/r/1430/</a>
</td>
</tr>
</table>
<br />
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Very nice work, where does this branch live? I'd like to start some testing with it.</pre>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/1430/diff/1/?file=20545#file20545line72" style="color: black; font-weight: bold; text-decoration: underline;">/asterisk/trunk/lib/python/asterisk/ThreadTestCondition.py</a>
<span style="font-weight: normal;">
(Diff revision 1)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">def evaluate(self, related_test_condition = None):</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">72</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">threads</span> <span class="o">=</span> <span class="n">ast</span><span class="o">.</span><span class="n">cli_exec</span><span class="p">(</span><span class="s">"core show threads"</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Again, future enhancement for Asterisk. Exposing this information via the AMI.</pre>
</div>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/1430/diff/1/?file=20546#file20546line34" style="color: black; font-weight: bold; text-decoration: underline;">/asterisk/trunk/lib/python/asterisk/buildoptions.py</a>
<span style="font-weight: normal;">
(Diff revision 1)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">def __init__(self, path=None):</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">34</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">buildopts_hdr_paths</span> <span class="o">=</span> <span class="p">[</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">35</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="s">"../include/asterisk/buildopts.h"</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">36</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="s">"/usr/include/asterisk/buildopts.h"</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">37</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="s">"/usr/local/include/asterisk/buildopts.h"</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">38</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="p">]</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Hmm... we should see about exposing this information directly in Asterisk, either the CLI / AMI or both. I'd like to get away from needing the actually asterisk source code as a dependency.
But, this maybe a future change too.</pre>
</div>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/1430/diff/1/?file=20551#file20551line10" style="color: black; font-weight: bold; text-decoration: underline;">/asterisk/trunk/tests/apps/voicemail/authenticate_nominal/test-config.yaml</a>
<span style="font-weight: normal;">
(Diff revision 1)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">10</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="l-Scalar-Plain">testconditions</span><span class="p-Indicator">:</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">11</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="p-Indicator">-</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">12</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="l-Scalar-Plain">name</span><span class="p-Indicator">:</span> <span class="s">'asterisk.ThreadTestCondition.ThreadPreTestCondition'</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">13</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="l-Scalar-Plain">type</span><span class="p-Indicator">:</span> <span class="s">'Pre'</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">14</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="p-Indicator">-</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">15</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="l-Scalar-Plain">name</span><span class="p-Indicator">:</span> <span class="s">'asterisk.ThreadTestCondition.ThreadPostTestCondition'</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">16</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="l-Scalar-Plain">type</span><span class="p-Indicator">:</span> <span class="s">'Post'</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">17</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="l-Scalar-Plain">relatedCondition</span><span class="p-Indicator">:</span> <span class="s">'ThreadPreTestCondition'</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is good for now, however once we are confident on well this logic is going to work, I would rather not have to define this for every tests.
This will help reduce the maintenance cost for tests.</pre>
</div>
<br />
<p>- Paul</p>
<br />
<p>On September 12th, 2011, 12:01 p.m., mjordan wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers and Paul Belanger.</div>
<div>By mjordan.</div>
<p style="color: grey;"><i>Updated Sept. 12, 2011, 12:01 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This patch provides a framework for doing pre- and post-test condition checking for an individual test run. This allows a test to take a snapshot of the state of its Asterisk instances before it executes and compare that against a snapshot of its Asterisk instances after the test executes. Depending on the test configuration, the test can either pass or fail based on whether or not resources were left uncollected in Asterisk after test execution.
The place in the test workflow where the conditions are evaluated is shown below:
main ConcreteTest TestClass TestConditionController
test.start_asterisk ----------------->start_asterisk
reactor.run -------------> run
|--------> run
|------------>evaluate_pre_checks()
|
test.stop_asterisk ------------------>stop_asterisk
|------------>evaluate_post_checks()
|
All pre- and post-test condition objects must inherit from the class TestCondition. The objects are created dynamically using python reflection and registered as either a pre- or post- check based on the test-config.yaml file for each test. Post TestConditions can be set to reference the Pre-Test condition object, such that the object is supplied to the post-test object during its evaluation. This allows the post-test condition object to make its evaluation based on the initial state of Asterisk as captured by the pre-test object. TestConditions are also allowed to have 'expected failure' states set to them, in case the tests in question are known to fail.
As part of this patch, a TestCondition set of objects were made to check for thread usage in Asterisk after a test. The objects ensure that after a test run, the threads currently active in Asterisk are the same threads that were active prior to the test run. The voicemail tests were updated to use this as an example.
Additionally, a few other enhancements were included with this patch:
1. A new class that allows tests to check what build options Asterisk was compiled with
2. Specifying the --test option now allows for a path to be specified as opposed to a specific test. All tests under that path will be executed, e.g., --test=tests/channels/SIP/ will execute all SIP tests
3. XML returned by runtests.py now includes ERROR log statements in the Failure message attribute
4. Test configuration was moved to its own class in the asterisk libraries. All TestClass-derived tests now have access to their configuration locally.
5. SIPpVersion was moved to the asterisk directory, as the other sipp related classes were already there and the TestConfig class in asterisk needed to reference it</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">trunk on local system.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/asterisk/trunk/lib/python/asterisk/ThreadTestCondition.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/buildoptions.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestState.py <span style="color: grey">(2176)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestConditions.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestConfig.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestCase.py <span style="color: grey">(2176)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/sippversion.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/sipp/version.py <span style="color: grey">(2090)</span></li>
<li>/asterisk/trunk/runtests.py <span style="color: grey">(2090)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/authenticate_nominal/run-test <span style="color: grey">(2177)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/authenticate_nominal/test-config.yaml <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/configs/ast1/extensions.conf <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/run-test <span style="color: grey">(2176)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/test-config.yaml <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_nominal/configs/ast1/extensions.conf <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_nominal/test-config.yaml <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/func_vmcount/configs/ast1/extensions.conf <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/func_vmcount/test-config.yaml <span style="color: grey">(2094)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_nominal/test-config.yaml <span style="color: grey">(2094)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/1430/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>