<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/2065/">https://reviewboard.asterisk.org/r/2065/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On November 20th, 2012, 2:39 p.m., <b>Matt Jordan</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/2065/diff/3/?file=30836#file30836line119" style="color: black; font-weight: bold; text-decoration: underline;">/asterisk/trunk/lib/python/asterisk/BridgeTestCase.py</a>
<span style="font-weight: normal;">
(Diff revision 3)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">def start_test(self, test_run):</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">119</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="bp">self</span><span class="o">.</span><span class="n">ami_alice</span><span class="o">.</span><span class="n">originate</span><span class="p">(</span><span class="n">channel</span> <span class="o">=</span> <span class="n">test_run</span><span class="p">[</span><span class="s">'originate_channel'</span><span class="p">],</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">120</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">exten</span> <span class="o">=</span> <span class="s">'test_call'</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">121</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">context</span> <span class="o">=</span> <span class="s">'default'</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">122</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">priority</span> <span class="o">=</span> <span class="s">'1'</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">123</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="n">variable</span> <span class="o">=</span> <span class="p">{</span><span class="s">'TALK_AUDIO'</span> <span class="p">:</span> <span class="s">'</span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getcwd</span><span class="p">(),</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">124</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="s">'lib/python/asterisk/audio'</span><span class="p">)})</span></pre></td>
</tr>
</tbody>
</table>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I'm not sure if you'll have any tests the won't Answer the originated channel. If you do, then don't do this.
Otherwise, attach an error handler to this and handle the originate failure. TestCase provides a convenience method to do this.
This comment applies to any channel originates.</pre>
</blockquote>
</blockquote>
<pre style="margin-left: 1em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">The nature of bridge tests is that they occur once a call is answered, so I can't foresee any case where the call will go unanswered.</pre>
<br />
<p>- Mark</p>
<br />
<p>On August 6th, 2012, 5:01 p.m., Mark Michelson wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers and Matt Jordan.</div>
<div>By Mark Michelson.</div>
<p style="color: grey;"><i>Updated Aug. 6, 2012, 5:01 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is the initial work towards writing a bridging test object, along with three sample tests.
The bridging test object works by starting three instances of Asterisk
Instance 0: the UUT
Instance 1: "Alice"
Instance 2: "Bob"
The basic setup is that Alice is the caller, Bob is the callee, and the UUT bridges the call.
In an attempt to make test creation simple, common configuration for the Asterisk instances is stored in the configs/bridge/ directory of the test suite. This way, it is not necessary to create duplicate configuration files for every test. Configuration files that are stored there at the moment are:
Alice:
extensions.conf
sip.conf
Bob:
extensions.conf
sip.conf
UUT:
features.conf
sip.conf
With these in place, the only file that typically needs to be written for each test is extensions.conf for the UUT.
YAML configuration:
The YAML configuration for the bridge test object is surprisingly simple. The configuration consists of a list of test calls to execute. For each call, there are three things to configure:
* Extension - This is the extension in the UUT's dialplan that Alice should dial. All extensions dialed are in the default context.
* Hangup - This lets the test object know whether it should send an AMI hangup to Alice or Bob. This option can be omitted if the hangup should not be initiated by the test object.
* Features - This is a list of DTMF features to execute once the call is established. Features consist of three options:
* Who - This lets the test object know whether Alice or Bob should initiate the feature
* What - This lets the test object know which feature to execute. The names of the features are the same as they would appear in features.conf
* Success - This lets the test object know whether the feature should succeed. "Succeeding" in this case simply meaans that the feature is recognized and that the proper built-in feature is executed. For evaluating success, we do not care about whether the feature actuall succeeded (e.g. automon actually successfully recorded audio)
Test Execution:
The YAML configuration is suggestive of how the test suite operates, but there are a few extra steps that may not seem obvious. Here are the steps executed when running a test:
1. Asterisk instances are started up. Base configuration in configs/bridge/ is installed first and test-specific configuration is applied afterward. This allows individual tests to overwrite base configuration if it is necessary.
2. Calls are executed one-by one. The first step is to originate a call from Alice to the extension specified for the call.
3. Once the call is connected, Alice will play an audio file towards Bob.
4. Bob records the file and then plays it back to Alice.
5. Alice records the file and then uses BackgroundDetect in order to determine if there is audio present. If there is, this establishes that two-way audio is present for the call.
6. The test suite is notified at this point that the call is up and that audio is present or not present.
7. The test suite then uses a series of AMI GetVar actions to determine if the connected line values for Alice and Bob are correct. It also checks that the BRIDGEPEER channel variable is correct.
8. Once all of these tests have passed, then features are executed one-by-one. The first step is to use an AMI PlayDTMF action in order to send the proper feature code from the correct Asterisk instance.
9. From here, the test suite waits for a TestEvent from Asterisk to determine if the feature succeeded or failed.
10. Once all features are finished, the next call is executed.
11. Once all calls are finished, the test is over.
Problems and potential improvements:
A problem due to the way review board works is that you can't see the test events I added into Asterisk without creating a separate review. I added a FEATURE_DETECTION test event that gives a "success" or "fail" result. The success result occurs after a feature callback has executed. The fail result occurs if, when checking for a feature, no feature is found that matches the DTMF sequence input.
One thing you'll notice while looking at the tests' extensions.conf files is that odd features have been enabled for certain parties. To give an example, in the disconnect test, the alice_disconnect extension gives Alice the permission to use the disconnect feature, which makes sense given the test. It also has given Bob the transfer permission, which may seem odd. This is because I had to work around an optimization in Asterisk. If a party has no DTMF features available to it, then the bridge will never get broken when DTMF is received from that party; the DTMF is always immediately passed through. This means that my test events in features.c would never trigger. My attempts to work around this resulted in failure, so for now I have gone with the odd configuration.
One problem that may be unavoidable is the length of time required for tests. Since each call requires three playbacks of a file, plus whatever time it takes to execute features, it means that tests can take a long time to execute. On my machine, the atxfer test takes around 60 seconds to complete. I at first attempted to verify audio on bridges using simpler methods, but they invariably failed for one reason or another.
One potential improvement that will need to be added is the ability to register observers for either mid-call or after a call has completed. Mid-call observers will be useful for testing things such as the L option for app_dial or for testing mid-call updates to connected line. I can't think of an immediate use for post-call observers, but having the facility available can't be a bad thing, right?
Another improvement that should be made is to make use of the CDRModule and CELModule in each test so that records can be verified.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">The three tests included here all pass. They are intended to illustrate variables in the test
simplebridge - A simple bridge test with no features. Illustrates that features are not required. Tests hangup of both the Alice and Bob Asterisk instances.
disconnect - Tests the disconnect feature. Tests that a sequence of features works as expected, both for successful and unsuccessful features. Demonstrates that tests with no hangup parameters work properly.
atxfer - Tests the atxfer feature. Similar to the disconnect feature except that a hangup is required from the test object.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/asterisk/trunk/configs/bridge/ast1/iax.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast1/sip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast2/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast2/iax.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast2/sip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast3/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast3/iax.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/configs/bridge/ast3/sip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/BridgeTestCase.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestCase.py <span style="color: grey">(3409)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/asterisk.py <span style="color: grey">(3409)</span></li>
<li>/asterisk/trunk/sample-yaml/ami-config.yaml.sample <span style="color: grey">(3409)</span></li>
<li>/asterisk/trunk/sample-yaml/bridge-config.yaml.sample <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/bridge/atxfer/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/bridge/atxfer/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/bridge/disconnect/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/bridge/disconnect/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/bridge/simple_bridge/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/bridge/simple_bridge/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/tests.yaml <span style="color: grey">(3409)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/2065/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>