<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/2013/">https://reviewboard.asterisk.org/r/2013/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On July 10th, 2012, 9:37 a.m., <b>Matt Jordan</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/2013/diff/3/?file=29956#file29956line191" style="color: black; font-weight: bold; text-decoration: underline;">/asterisk/team/mmichelson/bridge-tests/lib/python/asterisk/TestRunner.py</a>
<span style="font-weight: normal;">
(Diff revision 3)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">def create_test_object(test_path, test_config):</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">187</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="k">if</span> <span class="p">(</span><span class="s">'load-from-test'</span> <span class="ow">in</span> <span class="n">test_object_spec</span> <span class="ow">and</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">188</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">test_object_spec</span><span class="p">[</span><span class="s">'load-from-test'</span><span class="p">]):</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">189</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">TestModuleFinder</span><span class="o">.</span><span class="n">supported_paths</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">test_path</span><span class="p">)</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">190</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">test_path</span><span class="p">)</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">191</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
</tr>
</tbody>
</table>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">It seems a little odd to me that we're specifying 'load-from-test' in the Test Object configuration, when in reality that will affect the other loadable modules, not the test object. In fact, the test object has already been loaded into memory before this is parsed.
Maybe we should just change this configuration option to be 'search-in-test' or something that implies that the modules can be located in the test directory, as well as lib.python.asterisk?</pre>
</blockquote>
<p>On July 10th, 2012, 9:54 a.m., <b>Mark Michelson</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Yeah, load-from-test doesn't really pertain to the test-object itself, but more the test overall.
I didn't like handling it in the module configuration because it may be (and likely will be once bridging tests start being written) that multiple modules may be located in the test directory. I didn't like the idea of having:
test-modules:
test-object:
typename: 'SimpleTestCase.SimpleTestCase'
modules:
-
load-from-test: 'True'
typename: 'Alfa.Bravo'
-
load-from-test: 'True'
typename: 'Charlie.Delta'
The two load-from-test lines are redundant. Instead, the whole test should be told that loading from the test path is true. Perhaps the 'load-from-test' or 'search-in-test' option should be an option on the level of 'test-modules' and not inside the test-object or modules configuration?
Consider something like this:
test-modules:
load-from-test: 'True'
test-object:
typename: 'SimpleTestCase.SimpleTestCase'
modules:
-
typename: 'Alfa.Bravo'
-
typename: 'Charlie.Delta'
I also feel like 'load-from-path' should be up for the same consideration.</pre>
</blockquote>
</blockquote>
<pre style="margin-left: 1em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">How about something like this:
test-modules:
add-test-to-search-path: 'True'
add-to-search-path: 'tests/channels/foo/bar/'
add-to-search-path: 'my_random_folder/foo/bar/'
test-object:
typename: 'SimpleTestCase.SimpleTestCase'
modules:
-
typename: 'Alfa.Bravo'
-
typename: 'Charlie.Delta'</pre>
<br />
<p>- Matt</p>
<br />
<p>On July 10th, 2012, 9:16 a.m., Mark Michelson wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers and Matt Jordan.</div>
<div>By Mark Michelson.</div>
<p style="color: grey;"><i>Updated July 10, 2012, 9:16 a.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is a new pluggable module for the Asterisk test suite intended to be used to verify AMI events.
There are two ways that this may be used.
* "headermatch": This is the simpler of the two methods. When user-specified conditions are met, then headers in an AMI event are checked for expected values. If the check fails, then the test fails as well.
* "callback": For cases where more complicated checks are required than just AMI headers, then a user-defined callback may be called instead. This allows for the user to maintain their own test state to determine if the test has passed or failed.
There are some changes made outside the AMI module as well.
* TestCase has been updated to have a set_passed() method. Instead of directly modifying the 'passed' field of a test object, this method should be called. This way, if one module fails, other modules cannot override the failed state of the test. The CDR and ForkCDR modules have been updated accordingly
* TestRunner has been modified to get rid of the "load-from-test" option. Instead, when module configuration is loaded, it automatically loads the test path into its loader's supported paths. This was done to get around a situation where I wanted the test path added to the supported paths but did not want the TestRunner to actually load the module in that path.
I have also included a sample YAML file that details the options available for the AMI module.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I tested this by tweaking the cdr_userfield test with different values. The only option that is not thoroughly tested is the 'id' field since SimpleTestCase only uses a single Asterisk instance. The intention, once this gets committed, is to use this in a battery of Bridging tests that will be written in the near future.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/asterisk/team/mmichelson/bridge-tests/lib/python/asterisk/asterisk.py <span style="color: grey">(3290)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/lib/python/asterisk/TestCase.py <span style="color: grey">(3290)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/lib/python/asterisk/TestRunner.py <span style="color: grey">(3290)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/lib/python/asterisk/ami.py <span style="color: grey">(3290)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/lib/python/asterisk/cdr.py <span style="color: grey">(3290)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/sample-yaml/ami-config.yaml.sample <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/tests/cdr/ForkCdrModule.py <span style="color: grey">(3290)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/tests/predial/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/tests/predial/configs/ast1/followme.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/tests/predial/run-test <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/tests/predial/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/team/mmichelson/bridge-tests/tests/tests.yaml <span style="color: grey">(3290)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/2013/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>