<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/3966/">https://reviewboard.asterisk.org/r/3966/</a>
</td>
</tr>
</table>
<br />
<table bgcolor="#e0e0e0" width="100%" cellpadding="8" style="border: 1px gray solid;">
<tr>
<td>
<h1 style="margin-right: 0.2em; padding: 0; font-size: 10pt;">This change has been marked as submitted.</h1>
</td>
</tr>
</table>
<br />
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers.</div>
<div>By Mark Michelson.</div>
<p style="color: grey;"><i>Updated Sept. 18, 2014, 2:31 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Changes</h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Committed in revision 5603</pre>
</td>
</tr>
</table>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="https://issues.asterisk.org/jira/browse/ASTERISK-23874">ASTERISK-23874</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
testsuite
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is a suite of tests for resource list subscriptions with batched notification. Batched notification is a process by which a time interval may be configured on a resource list. When a state change occurs for one of the resources in the list, a timer is started. While the timer is running, any further state changes to any of the resources in the list are cached. Once the timer has expired, all state changes that have occurred for resources in the list are sent in a single NOTIFY. This set of tests means to ensure that the feature works as expected. The tests are as follows:
* basic: A baseline test, this creates a subscription to a resource list with two resources. A change is made to one of the resources, and we ensure that the notification of the state change occurs after the configured interval. This test also includes a resubscription and termination, ensuring that the NOTIFYs that we send in response to SUBSCRIBEs are sent immediately and do not use the configured interval.
* single_resource_multiple_changes: In this test, a single resource has two rapid state changes made on it. We ensure that even though we have changed the state twice, we only receive a single NOTIFY from Asterisk with the most recent state change.
* multiple_resources_single_change: In this test, two resources each have a single state change made on them. We ensure that even though we have changed two different resources' states, we receive only a single NOTIFY from Asterisk containing both resources' state changes.
* resubscription_interruption: In this test, we make a state change on a resource, and then immediately resubscribe to the list. Since we must send a NOTIFY immediately in response to the SUBSCRIBE, and since this NOTIFY will reflect the latest state of the resource, we test that the batched notification that we created when the state change occurred has been canceled. We do this by waiting for several seconds after the conclusion of the test to be certain that Asterisk does not send any errant NOTIFY requests.
* termination_interruption: This test is nearly identical to resubscription_interruption, except that instead of sending a resubscription to interrupt the batched notification, we send a subscription termination. Like with resubscription_interruption, we wait around after the scenario has completed in case Asterisk sends any further NOTIFY requests.
* list_of_lists/batched: This is the only test for lists containing sublists. In this scenario, the outer list has batched notifications disabled, but the inner sublist has batched notifications enabled. The test ensures that the configuration for notification batching on the inner list is ignored and that notifications are not batched at all. We do this by sending two rapid state changes and ensuring that each results in a NOTIFY request being sent by Asterisk.
In order to get these tests to work properly, I made a couple of changes to rls_test.py:
* I added a "stop_after_notifys" option, True by default. For the resubscription_interruption and termination_interruption tests, I needed rls_test.py not to stop the test after the final expected NOTIFY was received. This option allows those tests to keep the test alive until they deem it okay to stop the test.
* I made the call to register_scenario_started_callback() conditional. I did this initially because I was not going to use the SIPpTestCase for resubscription_interruption and termination_interruption. When not using the SIPpTestCase, an exception is thrown when trying to call register_scenario_started_callback() since it does not exist. In the end, I did end up using the SIPpTestCase for those tests, but I decided to leave the change to rls_test.py intact since it shouldn't necessarily presume that a SIPpTestCase is being used.
There are two versions of each of the tests, one for presence and one for MWI. For the most part, the tests are identical except for the obvious bits (Updating MWI instead of device state in test drivers, configuring mailboxes instead of hints, etc.). However, the single_resource_multiple_state_changes test is a bit different between the presence and MWI versions. This is because while testing with presence, I discovered issue https://issues.asterisk.org/jira/browse/ASTERISK-24286 . For presence tests, I could not reliably send two rapid state changes and expect consistent results. So instead, I have to send a single state change, wait for an AMI event telling me that Asterisk processed the state change, and then send the second state change. For MWI, the referenced issue is not a problem, so I can just send two MWI state changes rapidly and get consistent results.
As a final note, in order to properly detect when Asterisk sent NOTIFYs, and to know the exact nature of them, I added some test events to Asterisk. These can be found on /r/3967</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Ran all tests many many times to ensure they consistently pass.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/rls_test.py <span style="color: grey">(5533)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/tests.yaml <span style="color: grey">(5533)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/tests.yaml <span style="color: grey">(5533)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/tests.yaml <span style="color: grey">(5533)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/tests.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/configs/ast1/extensions.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/tests.yaml <span style="color: grey">(5533)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/tests.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/sipp/subscribe.xml <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/driver.py <span style="color: grey">(PRE-CREATION)</span></li>
<li>/asterisk/trunk/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/configs/ast1/pjsip.conf <span style="color: grey">(PRE-CREATION)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/3966/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>