[asterisk-commits] subscriptions/rls: Race conditions in some of the batched tests (testsuite[master])

SVN commits to the Asterisk project asterisk-commits at lists.digium.com
Tue Feb 14 07:17:09 CST 2017


Anonymous Coward #1000019 has submitted this change and it was merged. ( https://gerrit.asterisk.org/4923 )

Change subject: subscriptions/rls: Race conditions in some of the batched tests
......................................................................


subscriptions/rls: Race conditions in some of the batched tests

Several of the rls list and list of lists mwi and presence batched tests could
fail due to a race between the rls_test and driver pluggable modules. Most of
the patched tests had multiple modules that could stop the running test and
depending on which one stop the test first some of the test conditions may not
have been checked yet.

First, all tests were initiated by the sipp test module. If the 'stop-after-
scenarios' option was not set to false then once the sipp scenario ended it
would stop the test. This patch sets the option to false on the offending
tests. It also removes any pause statements from the sipp scenario that were
there only to keep the scenario from ending and stopping the test too soon.

Second, the rls_test pluggable module would also stop the test once all its
configured conditions had been met. It too contains an option, 'stop-test-
after-notifys', that when set to false will not stop the test after validating
conditions. However, for several of the tests this option was not set and as it
defaults to true the test would sometimes end early. This patch sets that option
to false for the tests that needed it.

Third, all the problematic tests have/had a driver pluggable module that also
check against certain conditions and then would either stop the test too, or
would wait until one of the other modules mentioned above would. Again, though
if one of the ones above stopped the test before meeting the conditions of this
module then the test would fail. Or in the cases where this module stops the
test first the others may not have finished.

In order to alleviate any race conditions and also make tests deterministic,
all tests were given one stopping point on success. Some of the tests just
needed the options mentioned above set appropriately. Others required the
driver module to be removed and their conditions are instead checked by using
the EventActionModule. For these an UserEvent is now raised once rls_test is
done. The EventActionModule can then wait for that event and the other ami
conditional events before stopping.

Change-Id: I7c95ce6b8d265020cb12dc033ecc9234b522f7e2
---
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/driver.py
M tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/driver.py
M tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/rls_test.py
23 files changed, 371 insertions(+), 610 deletions(-)

Approvals:
  George Joseph: Looks good to me, but someone else must approve
  Anonymous Coward #1000019: Verified
  Joshua Colp: Looks good to me, approved



diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml
index bf8c68a..308295e 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml
@@ -46,12 +46,14 @@
             typename: 'driver.TestDriver'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'mail_list'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'mail_list'
     packets:
         -
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml
index 2b364bc..8c3e4cd 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml
@@ -40,12 +40,14 @@
             typename: 'driver.TestDriver'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'mail_list'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'mail_list'
     packets:
         -
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/driver.py b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/driver.py
deleted file mode 100755
index 01cef92..0000000
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/driver.py
+++ /dev/null
@@ -1,95 +0,0 @@
-#!/usr/bin/env python
-
-import logging
-
-LOGGER = logging.getLogger(__name__)
-
-# These are the states that are moved through during the tests. They are listed
-# here in the order that they occur throughout the test.
-
-# Initial state before subscription establishment.
-UNESTABLISHED = 1
-
-# This state is entered after initial SUBSCRIBE-NOTIFY exchange has occurred.
-ESTABLISHED = 2
-
-# This state is entered after the SUBSCRIBE-NOTIFY exchange for subscription
-# refresh has occurred.
-REFRESHED = 3
-
-
-class TestDriver(object):
-    def __init__(self, module_config, test_object):
-        self.ami = None
-        self.state = UNESTABLISHED
-        self.test_object = test_object
-        self.test_object.register_ami_observer(self.ami_connect)
-        self.test_object.register_scenario_stopped_observer(
-            self.on_scenario_complete)
-
-    def fail_test(self):
-        self.test_object.set_passed(False)
-        self.test_object.stop_reactor()
-
-    def pass_test(self):
-        self.test_object.set_passed(True)
-        self.test_object.stop_reactor()
-
-    def on_scenario_complete(self, result):
-        if not result.passed:
-            LOGGER.error("SIPp scenario failed")
-            self.fail_test()
-
-        if self.state != REFRESHED:
-            LOGGER.error("SIPp scenario finished with test in unexpected "
-                         "state. Expected {0} but was in {1}".format(
-                             REFRESHED, self.state))
-            self.fail_test()
-
-    def ami_connect(self, ami):
-        self.ami = ami
-        self.ami.registerEvent('TestEvent', self.on_test_event)
-
-    def on_test_event(self, ami, event):
-        state = event['state']
-        if state == 'SUBSCRIPTION_ESTABLISHED':
-            self.on_subscription_established()
-        elif state == 'SUBSCRIPTION_REFRESHED':
-            self.on_subscription_refreshed()
-        elif state == 'SUBSCRIPTION_TERMINATED':
-            self.on_subscription_terminated()
-        elif state == 'SUBSCRIPTION_STATE_CHANGED':
-            self.on_subscription_state_change()
-
-    def on_subscription_established(self):
-        if self.state != UNESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, UNESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(ESTABLISHED))
-        self.state = ESTABLISHED
-        message = {
-            'Action': 'MWIUpdate',
-            'Mailbox': 'alice',
-            'NewMessages': '1',
-            'OldMessages': '0'
-        }
-        self.ami.sendMessage(message)
-
-    def on_subscription_state_change(self):
-        LOGGER.error("Unexpected state change NOTIFY received")
-        self.fail_test()
-
-    def on_subscription_refreshed(self):
-        if self.state != ESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, ESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(REFRESHED))
-        self.state = REFRESHED
-
-    def on_subscription_terminated(self):
-        LOGGER.error("Unexpected subscription termination received")
-        self.fail_test()
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml
index 8e5d7a8..8a48007 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml
@@ -114,7 +114,4 @@
 
     ]]>
   </send>
-
-  <timewait milliseconds="8000" />
-
 </scenario>
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml
index 27d790c..da9bcce 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml
@@ -6,8 +6,6 @@
          * A state change occurs on one of the resources in the list.
          * While the state change is being batched by Asterisk, the SIPp scenario resubscribes.
          * The resubscription causes an immediate NOTIFY to be sent.
-         * The SIPp scenario then waits for a few seconds to ensure that the batched notification
-           has been canceled.'
 
 properties:
     minversion: '13.0.0'
@@ -39,9 +37,11 @@
             config-section: 'test-config'
             typename: 'rls_test.RLSTest'
         -
-            typename: 'driver.TestDriver'
+            config-section: 'ami-config'
+            typename: 'pluggable_modules.EventActionModule'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
@@ -57,3 +57,58 @@
         -
             full-state: True
             resources: { 'alice': {'type': 'MWI', 'state': 'active', 'voice_message': '1/0 (0/0)', 'messages_waiting': 'yes'}, 'bob': {'type': 'MWI', 'state': 'active', 'voice_message': '0/0 (0/0)', 'messages_waiting': 'no'} }
+
+ami-config:
+    -
+        ami-events:
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_ESTABLISHED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'MWIUpdate'
+                Mailbox: 'alice'
+                NewMessages: '1'
+                OldMessages: '0'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_STATE_CHANGED'
+            count: 0 # shouldn't receive any of these
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_REFRESHED'
+            count: 1
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_TERMINATED'
+            count: 0 # shouldn't receive any of these
+        ami-actions:
+          -
+            action:
+                Action: 'UserEvent'
+                UserEvent: 'RLSResult'
+                Result: 'pass'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'UserEvent'
+                    UserEvent: 'RLSResult'
+            requirements:
+                match:
+                    Result: 'pass'
+            count: 2 # one after expected test events have been received, one after validation
+            trigger-on-count: True
+        stop_test:
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml
index 126c0a2..844d47d 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml
@@ -40,12 +40,14 @@
             typename: 'driver.TestDriver'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'mail_list'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'mail_list'
     packets:
         -
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/driver.py b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/driver.py
deleted file mode 100755
index 377c33d..0000000
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/driver.py
+++ /dev/null
@@ -1,96 +0,0 @@
-#!/usr/bin/env python
-
-import logging
-
-LOGGER = logging.getLogger(__name__)
-
-# These are the states that are moved through during the tests. They are listed
-# here in the order that they occur throughout the test.
-
-# Initial state before subscription establishment.
-UNESTABLISHED = 1
-
-# This state is entered after initial SUBSCRIBE-NOTIFY exchange has occurred.
-ESTABLISHED = 2
-
-# This state is entered after SUBSCRIBE-NOTIFY exchange to terminate
-# subscription has occurred.
-TERMINATED = 3
-
-
-class TestDriver(object):
-    def __init__(self, module_config, test_object):
-        self.ami = None
-        self.state = UNESTABLISHED
-        self.test_object = test_object
-        self.test_object.register_ami_observer(self.ami_connect)
-        self.test_object.register_scenario_stopped_observer(
-            self.on_scenario_complete)
-
-    def fail_test(self):
-        self.test_object.set_passed(False)
-        self.test_object.stop_reactor()
-
-    def pass_test(self):
-        self.test_object.set_passed(True)
-        self.test_object.stop_reactor()
-
-    def on_scenario_complete(self, result):
-        if not result.passed:
-            LOGGER.error("SIPp scenario failed")
-            self.fail_test()
-
-        if self.state != TERMINATED:
-            LOGGER.error("SIPp scenario finished with test in unexpected "
-                         "state. Expected {0} but was in {1}".format(
-                             TERMINATED, self.state))
-            self.fail_test()
-
-    def ami_connect(self, ami):
-        self.ami = ami
-        self.ami.registerEvent('TestEvent', self.on_test_event)
-
-    def on_test_event(self, ami, event):
-        state = event['state']
-        if state == 'SUBSCRIPTION_ESTABLISHED':
-            self.on_subscription_established()
-        elif state == 'SUBSCRIPTION_REFRESHED':
-            self.on_subscription_refreshed()
-        elif state == 'SUBSCRIPTION_TERMINATED':
-            self.on_subscription_terminated()
-        elif state == 'SUBSCRIPTION_STATE_CHANGED':
-            self.on_subscription_state_change()
-
-    def on_subscription_established(self):
-        if self.state != UNESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, UNESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(ESTABLISHED))
-        self.state = ESTABLISHED
-        message = {
-            'Action': 'MWIUpdate',
-            'Mailbox': 'alice',
-            'NewMessages': '1',
-            'OldMessages': '0'
-        }
-        self.ami.sendMessage(message)
-
-    def on_subscription_state_change(self):
-        LOGGER.error("Unexpected state change NOTIFY received")
-        self.fail_test()
-
-    def on_subscription_refreshed(self):
-        LOGGER.error("Unexpected resubscription received")
-        self.fail_test()
-
-    def on_subscription_terminated(self):
-        if self.state != ESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, ESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(TERMINATED))
-        self.state = TERMINATED
-
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml
index e58f921..cb5f060 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml
@@ -114,7 +114,4 @@
 
     ]]>
   </send>
-
-  <timewait milliseconds="8000" />
-
 </scenario>
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml
index 833e906..fb3831f 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml
@@ -6,8 +6,6 @@
          * A state change occurs on one of the resources in the list.
          * While the state change is being batched by Asterisk, the SIPp scenario terminates the subscription.
          * The subscription termination causes an immediate NOTIFY to be sent.
-         * The SIPp scenario then waits for a few seconds to ensure that the batched notification
-           has been canceled.'
 
 properties:
     minversion: '13.0.0'
@@ -39,9 +37,11 @@
             config-section: 'test-config'
             typename: 'rls_test.RLSTest'
         -
-            typename: 'driver.TestDriver'
+            config-section: 'ami-config'
+            typename: 'pluggable_modules.EventActionModule'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
@@ -57,3 +57,58 @@
         -
             full-state: True
             resources: { 'alice': {'type': 'MWI', 'state': 'terminated', 'voice_message': '1/0 (0/0)', 'messages_waiting': 'yes'}, 'bob': {'type': 'MWI', 'state': 'terminated', 'voice_message': '0/0 (0/0)', 'messages_waiting': 'no'} }
+
+ami-config:
+    -
+        ami-events:
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_ESTABLISHED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'MWIUpdate'
+                Mailbox: 'alice'
+                NewMessages: '1'
+                OldMessages: '0'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_STATE_CHANGED'
+            count: 0 # shouldn't receive any of these
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_REFRESHED'
+            count: 0 # shouldn't receive any of these
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_TERMINATED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'UserEvent'
+                UserEvent: 'RLSResult'
+                Result: 'pass'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'UserEvent'
+                    UserEvent: 'RLSResult'
+            requirements:
+                match:
+                    Result: 'pass'
+            count: 2 # one after expected test events have been received, one after validation
+            trigger-on-count: True
+        stop_test:
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml
index abe484a..f00bf9e 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml
@@ -44,6 +44,7 @@
             typename: 'driver.TestDriver'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml
index 6ec511b..f3e2536 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml
@@ -36,12 +36,14 @@
             typename: 'driver.TestDriver'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'pres_list'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'pres_list'
     packets:
         -
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/driver.py b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/driver.py
deleted file mode 100755
index ae80dee..0000000
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/driver.py
+++ /dev/null
@@ -1,94 +0,0 @@
-#!/usr/bin/env python
-
-import logging
-
-LOGGER = logging.getLogger(__name__)
-
-# These are the states that are moved through during the tests. They are listed
-# here in the order that they occur throughout the test.
-
-# Initial state before subscription establishment.
-UNESTABLISHED = 1
-
-# This state is entered after initial SUBSCRIBE-NOTIFY exchange has occurred.
-ESTABLISHED = 2
-
-# This state is entered after the SUBSCRIBE-NOTIFY exchange for subscription
-# refresh has occurred.
-REFRESHED = 3
-
-
-class TestDriver(object):
-    def __init__(self, module_config, test_object):
-        self.ami = None
-        self.state = UNESTABLISHED
-        self.test_object = test_object
-        self.test_object.register_ami_observer(self.ami_connect)
-        self.test_object.register_scenario_stopped_observer(
-            self.on_scenario_complete)
-
-    def fail_test(self):
-        self.test_object.set_passed(False)
-        self.test_object.stop_reactor()
-
-    def pass_test(self):
-        self.test_object.set_passed(True)
-        self.test_object.stop_reactor()
-
-    def on_scenario_complete(self, result):
-        if not result.passed:
-            LOGGER.error("SIPp scenario failed")
-            self.fail_test()
-
-        if self.state != REFRESHED:
-            LOGGER.error("SIPp scenario finished with test in unexpected "
-                         "state. Expected {0} but was in {1}".format(
-                             REFRESHED, self.state))
-            self.fail_test()
-
-    def ami_connect(self, ami):
-        self.ami = ami
-        self.ami.registerEvent('TestEvent', self.on_test_event)
-
-    def on_test_event(self, ami, event):
-        state = event['state']
-        if state == 'SUBSCRIPTION_ESTABLISHED':
-            self.on_subscription_established()
-        elif state == 'SUBSCRIPTION_REFRESHED':
-            self.on_subscription_refreshed()
-        elif state == 'SUBSCRIPTION_TERMINATED':
-            self.on_subscription_terminated()
-        elif state == 'SUBSCRIPTION_STATE_CHANGED':
-            self.on_subscription_state_change()
-
-    def on_subscription_established(self):
-        if self.state != UNESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, UNESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(ESTABLISHED))
-        self.state = ESTABLISHED
-        message = {
-            'Action': 'SetVar',
-            'Variable': 'DEVICE_STATE(Custom:alice)',
-            'Value': 'InUse'
-        }
-        self.ami.sendMessage(message)
-
-    def on_subscription_state_change(self):
-        LOGGER.error("Unexpected state change NOTIFY received")
-        self.fail_test()
-
-    def on_subscription_refreshed(self):
-        if self.state != ESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, ESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(REFRESHED))
-        self.state = REFRESHED
-
-    def on_subscription_terminated(self):
-        LOGGER.error("Unexpected subscription termination received")
-        self.fail_test()
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml
index b181548..10b8a07 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml
@@ -114,7 +114,4 @@
 
     ]]>
   </send>
-
-  <timewait milliseconds="8000" />
-
 </scenario>
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml
index 6a51b4b..b51f288 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml
@@ -6,8 +6,6 @@
          * A state change occurs on one of the resources in the list.
          * While the state change is being batched by Asterisk, the SIPp scenario resubscribes.
          * The resubscription causes an immediate NOTIFY to be sent.
-         * The SIPp scenario then waits for a few seconds to ensure that the batched notification
-           has been canceled.'
 
 properties:
     minversion: '13.0.0'
@@ -35,9 +33,11 @@
             config-section: 'test-config'
             typename: 'rls_test.RLSTest'
         -
-            typename: 'driver.TestDriver'
+            config-section: 'ami-config'
+            typename: 'pluggable_modules.EventActionModule'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
@@ -53,3 +53,57 @@
         -
             full-state: True
             resources: { 'alice': {'type': 'PIDF', 'state': 'active'}, 'bob': {'type': 'PIDF', 'state': 'active'}}
+
+ami-config:
+    -
+        ami-events:
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_ESTABLISHED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'SetVar'
+                Variable: 'DEVICE_STATE(Custom:alice)'
+                Value: 'InUse'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_STATE_CHANGED'
+            count: 0 # shouldn't receive any of these
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_REFRESHED'
+            count: 1
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_TERMINATED'
+            count: 0 # shouldn't receive any of these
+        ami-actions:
+          -
+            action:
+                Action: 'UserEvent'
+                UserEvent: 'RLSResult'
+                Result: 'pass'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'UserEvent'
+                    UserEvent: 'RLSResult'
+            requirements:
+                match:
+                    Result: 'pass'
+            count: 2 # one after expected test events have been received, one after validation
+            trigger-on-count: True
+        stop_test:
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml
index 45bde1f..bd9bc51 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml
@@ -36,12 +36,14 @@
             typename: 'driver.TestDriver'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'pres_list'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'pres_list'
     packets:
         -
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/driver.py b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/driver.py
deleted file mode 100755
index 7e7a2b7..0000000
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/driver.py
+++ /dev/null
@@ -1,95 +0,0 @@
-#!/usr/bin/env python
-
-import logging
-
-LOGGER = logging.getLogger(__name__)
-
-# These are the states that are moved through during the tests. They are listed
-# here in the order that they occur throughout the test.
-
-# Initial state before subscription establishment.
-UNESTABLISHED = 1
-
-# This state is entered after initial SUBSCRIBE-NOTIFY exchange has occurred.
-ESTABLISHED = 2
-
-# This state is entered after SUBSCRIBE-NOTIFY exchange to terminate
-# subscription has occurred.
-TERMINATED = 3
-
-
-class TestDriver(object):
-    def __init__(self, module_config, test_object):
-        self.ami = None
-        self.state = UNESTABLISHED
-        self.test_object = test_object
-        self.test_object.register_ami_observer(self.ami_connect)
-        self.test_object.register_scenario_stopped_observer(
-            self.on_scenario_complete)
-
-    def fail_test(self):
-        self.test_object.set_passed(False)
-        self.test_object.stop_reactor()
-
-    def pass_test(self):
-        self.test_object.set_passed(True)
-        self.test_object.stop_reactor()
-
-    def on_scenario_complete(self, result):
-        if not result.passed:
-            LOGGER.error("SIPp scenario failed")
-            self.fail_test()
-
-        if self.state != TERMINATED:
-            LOGGER.error("SIPp scenario finished with test in unexpected "
-                         "state. Expected {0} but was in {1}".format(
-                             TERMINATED, self.state))
-            self.fail_test()
-
-    def ami_connect(self, ami):
-        self.ami = ami
-        self.ami.registerEvent('TestEvent', self.on_test_event)
-
-    def on_test_event(self, ami, event):
-        state = event['state']
-        if state == 'SUBSCRIPTION_ESTABLISHED':
-            self.on_subscription_established()
-        elif state == 'SUBSCRIPTION_REFRESHED':
-            self.on_subscription_refreshed()
-        elif state == 'SUBSCRIPTION_TERMINATED':
-            self.on_subscription_terminated()
-        elif state == 'SUBSCRIPTION_STATE_CHANGED':
-            self.on_subscription_state_change()
-
-    def on_subscription_established(self):
-        if self.state != UNESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, UNESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(ESTABLISHED))
-        self.state = ESTABLISHED
-        message = {
-            'Action': 'SetVar',
-            'Variable': 'DEVICE_STATE(Custom:alice)',
-            'Value': 'InUse'
-        }
-        self.ami.sendMessage(message)
-
-    def on_subscription_state_change(self):
-        LOGGER.error("Unexpected state change NOTIFY received")
-        self.fail_test()
-
-    def on_subscription_refreshed(self):
-        LOGGER.error("Unexpected resubscription received")
-        self.fail_test()
-
-    def on_subscription_terminated(self):
-        if self.state != ESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, ESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(TERMINATED))
-        self.state = TERMINATED
-
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml
index e73aae6..06b1246 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml
@@ -114,7 +114,4 @@
 
     ]]>
   </send>
-
-  <timewait milliseconds="8000" />
-
 </scenario>
diff --git a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml
index 0b7488b..6e0e747 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml
@@ -6,8 +6,6 @@
          * A state change occurs on one of the resources in the list.
          * While the state change is being batched by Asterisk, the SIPp scenario terminates the subscription.
          * The subscription termination causes an immediate NOTIFY to be sent.
-         * The SIPp scenario then waits for a few seconds to ensure that the batched notification
-           has been canceled.'
 
 properties:
     minversion: '13.0.0'
@@ -35,9 +33,11 @@
             config-section: 'test-config'
             typename: 'rls_test.RLSTest'
         -
-            typename: 'driver.TestDriver'
+            config-section: 'ami-config'
+            typename: 'pluggable_modules.EventActionModule'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
@@ -53,3 +53,57 @@
         -
             full-state: True
             resources: { 'alice': {'type': 'PIDF', 'state': 'terminated'}, 'bob': {'type': 'PIDF', 'state': 'terminated'}}
+
+ami-config:
+    -
+        ami-events:
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_ESTABLISHED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'SetVar'
+                Variable: 'DEVICE_STATE(Custom:alice)'
+                Value: 'InUse'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_STATE_CHANGED'
+            count: 0 # shouldn't receive any of these
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_REFRESHED'
+            count: 0 # shouldn't receive any of these
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_TERMINATED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'UserEvent'
+                UserEvent: 'RLSResult'
+                Result: 'pass'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'UserEvent'
+                    UserEvent: 'RLSResult'
+            requirements:
+                match:
+                    Result: 'pass'
+            count: 2 # one after expected test events have been received, one after validation
+            trigger-on-count: True
+        stop_test:
diff --git a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/driver.py b/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/driver.py
deleted file mode 100755
index 2c94b3d..0000000
--- a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/driver.py
+++ /dev/null
@@ -1,100 +0,0 @@
-#!/usr/bin/env python
-
-import logging
-
-LOGGER = logging.getLogger(__name__)
-
-# These are the states that are moved through during the tests. They are listed
-# here in the order that they occur throughout the test.
-
-# Initial state before subscription establishment.
-UNESTABLISHED = 1
-
-# This state is entered after initial SUBSCRIBE-NOTIFY exchange has occurred.
-ESTABLISHED = 2
-
-# This state is entered after Alice's MWI NOTIFY has been received.
-ALICE_STATE = 3
-
-# This state is entered after Bob's MWI NOTIFY has been received.
-BOB_STATE = 4
-
-
-class TestDriver(object):
-    def __init__(self, module_config, test_object):
-        self.ami = None
-        self.state = UNESTABLISHED
-        self.test_object = test_object
-        self.scenario_completed = False
-        test_object.register_ami_observer(self.ami_connect)
-        test_object.register_scenario_stopped_observer(self.scenario_complete)
-
-    def fail_test(self):
-        self.test_object.set_passed(False)
-        self.test_object.stop_reactor()
-
-    def ami_connect(self, ami):
-        self.ami = ami
-        self.ami.registerEvent('TestEvent', self.on_test_event)
-
-    def scenario_complete(self, scenario):
-        if self.state != BOB_STATE:
-            LOGGER.error("Test ended on unexpected state {0}".format(self.state))
-            self.fail_test()
-
-        self.scenario_completed = True
-
-    def on_test_event(self, ami, event):
-        state = event['state']
-        if state == 'SUBSCRIPTION_ESTABLISHED':
-            self.on_subscription_established()
-        elif state == 'SUBSCRIPTION_REFRESHED':
-            self.on_subscription_refreshed()
-        elif state == 'SUBSCRIPTION_TERMINATED':
-            self.on_subscription_terminated()
-        elif state == 'SUBSCRIPTION_STATE_CHANGED':
-            self.on_subscription_state_change()
-
-    def on_subscription_established(self):
-        if self.state != UNESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, UNESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(ESTABLISHED))
-        self.state = ESTABLISHED
-        message = {
-            'Action': 'MWIUpdate',
-            'Mailbox': 'alice',
-            'NewMessages': '1',
-            'OldMessages': '0'
-        }
-        self.ami.sendMessage(message)
-        message['Mailbox'] = 'bob'
-        self.ami.sendMessage(message)
-
-    def on_subscription_state_change(self):
-        if self.state == ESTABLISHED:
-            self.state = ALICE_STATE
-            LOGGER.debug("State change to {0}".format(ALICE_STATE))
-        elif self.state == ALICE_STATE:
-            self.state = BOB_STATE
-            LOGGER.debug("State change to {0}".format(BOB_STATE))
-        else:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1} or {2}".format(self.state, ESTABLISHED,
-                                             ALICE_STATE))
-            self.fail_test()
-
-    def on_subscription_refreshed(self):
-        LOGGER.error("Unexpected resubscription")
-        self.fail_test()
-
-    def on_subscription_terminated(self):
-        # Asterisk will send a termination NOTIFY on shutdown. As long as this
-        # happens after the scenario completes, it's fine.
-        if self.scenario_completed:
-            return
-
-        LOGGER.error("Unexpected subscription termination")
-        self.fail_test()
diff --git a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml
index 33ea863..18ebc4b 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml
@@ -39,15 +39,18 @@
             config-section: 'test-config'
             typename: 'rls_test.RLSTest'
         -
-            typename: 'driver.TestDriver'
+            config-section: 'ami-config'
+            typename: 'pluggable_modules.EventActionModule'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'mail_list_a'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'mail_list_a'
     packets:
         -
@@ -78,3 +81,59 @@
                     }
                 }
             }
+
+ami-config:
+    -
+        ami-events:
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_ESTABLISHED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'MWIUpdate'
+                Mailbox: 'alice'
+                NewMessages: '1'
+                OldMessages: '0'
+          -
+            action:
+                Action: 'MWIUpdate'
+                Mailbox: 'bob'
+                NewMessages: '1'
+                OldMessages: '0'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_STATE_CHANGED'
+            count: 2 # one for alice, one for bob
+            trigger-on-count: True
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_REFRESHED'
+            count: 0 # shouldn't receive any of these
+        ami-actions:
+          -
+            action:
+                Action: 'UserEvent'
+                UserEvent: 'RLSResult'
+                Result: 'pass'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'UserEvent'
+                    UserEvent: 'RLSResult'
+            requirements:
+                match:
+                    Result: 'pass'
+            count: 2 # one after expected test events have been received, one after validation
+            trigger-on-count: True
+        stop_test:
diff --git a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/driver.py b/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/driver.py
deleted file mode 100755
index dfd6d3a..0000000
--- a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/driver.py
+++ /dev/null
@@ -1,99 +0,0 @@
-#!/usr/bin/env python
-
-import logging
-
-LOGGER = logging.getLogger(__name__)
-
-# These are the states that are moved through during the tests. They are listed
-# here in the order that they occur throughout the test.
-
-# Initial state before subscription establishment.
-UNESTABLISHED = 1
-
-# This state is entered after initial SUBSCRIBE-NOTIFY exchange has occurred.
-ESTABLISHED = 2
-
-# This state is entered after Alice's device state NOTIFY has been received.
-ALICE_STATE = 3
-
-# This state is entered after Bob's device state NOTIFY has been received.
-BOB_STATE = 4
-
-
-class TestDriver(object):
-    def __init__(self, module_config, test_object):
-        self.ami = None
-        self.state = UNESTABLISHED
-        self.test_object = test_object
-        self.scenario_completed = False
-        test_object.register_ami_observer(self.ami_connect)
-        test_object.register_scenario_stopped_observer(self.scenario_complete)
-
-    def fail_test(self):
-        self.test_object.set_passed(False)
-        self.test_object.stop_reactor()
-
-    def ami_connect(self, ami):
-        self.ami = ami
-        self.ami.registerEvent('TestEvent', self.on_test_event)
-
-    def scenario_complete(self, scenario):
-        if self.state != BOB_STATE:
-            LOGGER.error("Test ended on unexpected state {0}".format(self.state))
-            self.fail_test()
-
-        self.scenario_completed = True
-
-    def on_test_event(self, ami, event):
-        state = event['state']
-        if state == 'SUBSCRIPTION_ESTABLISHED':
-            self.on_subscription_established()
-        elif state == 'SUBSCRIPTION_REFRESHED':
-            self.on_subscription_refreshed()
-        elif state == 'SUBSCRIPTION_TERMINATED':
-            self.on_subscription_terminated()
-        elif state == 'SUBSCRIPTION_STATE_CHANGED':
-            self.on_subscription_state_change()
-
-    def on_subscription_established(self):
-        if self.state != UNESTABLISHED:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1}".format(self.state, UNESTABLISHED))
-            self.fail_test()
-
-        LOGGER.debug("State change to {0}".format(ESTABLISHED))
-        self.state = ESTABLISHED
-        message = {
-            'Action': 'SetVar',
-            'Variable': 'DEVICE_STATE(Custom:alice)',
-            'Value': 'InUse'
-        }
-        self.ami.sendMessage(message)
-        message['Variable'] = 'DEVICE_STATE(Custom:bob)'
-        self.ami.sendMessage(message)
-
-    def on_subscription_state_change(self):
-        if self.state == ESTABLISHED:
-            self.state = ALICE_STATE
-            LOGGER.debug("State change to {0}".format(ALICE_STATE))
-        elif self.state == ALICE_STATE:
-            self.state = BOB_STATE
-            LOGGER.debug("State change to {0}".format(BOB_STATE))
-        else:
-            LOGGER.error("Unexpected state change from {0}. Expected state "
-                         "{1} or {2}".format(self.state, ESTABLISHED,
-                                             ALICE_STATE))
-            self.fail_test()
-
-    def on_subscription_refreshed(self):
-        LOGGER.error("Unexpected resubscription")
-        self.fail_test()
-
-    def on_subscription_terminated(self):
-        # Asterisk will send a termination NOTIFY on shutdown. As long as this
-        # happens after the scenario completes, it's fine.
-        if self.scenario_completed:
-            return
-
-        LOGGER.error("Unexpected subscription termination")
-        self.fail_test()
diff --git a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml b/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml
index 7b536b7..b9eea9f 100644
--- a/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml
+++ b/tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml
@@ -35,15 +35,18 @@
             config-section: 'test-config'
             typename: 'rls_test.RLSTest'
         -
-            typename: 'driver.TestDriver'
+            config-section: 'ami-config'
+            typename: 'pluggable_modules.EventActionModule'
 
 test-case-config:
+    stop-after-scenarios: False
     test-iterations:
         -
             scenarios:
                 - { 'target': '127.0.0.1:5061', 'key-args': {'scenario': 'subscribe.xml', '-i': '127.0.0.1', '-p': '5062', '-s': 'pres_list_a'} }
 
 test-config:
+    stop-test-after-notifys: False
     list-name: 'pres_list_a'
     packets:
         -
@@ -74,3 +77,57 @@
                     }
                 }
             }
+
+ami-config:
+    -
+        ami-events:
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_ESTABLISHED'
+            count: 1
+        ami-actions:
+          -
+            action:
+                Action: 'SetVar'
+                Variable: 'DEVICE_STATE(Custom:alice)'
+                Value: 'InUse'
+          -
+            action:
+                Action: 'SetVar'
+                Variable: 'DEVICE_STATE(Custom:bob)'
+                Value: 'InUse'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_STATE_CHANGED'
+            count: 2 # one for alice, one for bob
+            trigger-on-count: True
+          -
+            conditions:
+                match:
+                    Event: 'TestEvent'
+                    State: 'SUBSCRIPTION_REFRESHED'
+            count: 0 # shouldn't receive any of these
+        ami-actions:
+          -
+            action:
+                Action: 'UserEvent'
+                UserEvent: 'RLSResult'
+                Result: 'pass'
+    -
+        ami-events:
+          -
+            conditions:
+                match:
+                    Event: 'UserEvent'
+                    UserEvent: 'RLSResult'
+            requirements:
+                match:
+                    Result: 'pass'
+            count: 2 # one after expected test events have been received, one after validation
+            trigger-on-count: True
+        stop_test:
diff --git a/tests/channels/pjsip/subscriptions/rls/rls_test.py b/tests/channels/pjsip/subscriptions/rls/rls_test.py
index 3364ee2..4ed9730 100755
--- a/tests/channels/pjsip/subscriptions/rls/rls_test.py
+++ b/tests/channels/pjsip/subscriptions/rls/rls_test.py
@@ -158,6 +158,14 @@
         packet                 -- Incoming SIP Packet
         """
 
+        def __mark_completed():
+            self.ami.userEvent('RLSResult', result='pass')
+            if self.stop_test_after_notifys:
+                # We only deal with as many NOTIFIES as we have defined in our
+                # test-config.yaml
+                self.test_object.set_passed(True)
+                self.test_object.stop_reactor()
+
         # We are accessing a pseudo-private member here publicly, which
         # shouldn't really be done. (Granted, read-only, hence why we can
         # do this and not break the world.)
@@ -216,11 +224,10 @@
             info_msg = "All test phases completed. RLS verification complete."
             LOGGER.info(info_msg)
             self.test_object.remove_fail_token(self.token)
-            if self.stop_test_after_notifys:
-                # We only deal with as many NOTIFIES as we have defined in our
-                # test-config.yaml
-                self.test_object.set_passed(True)
-                self.test_object.stop_reactor()
+            # Notify that we are done - give a couple of seconds for the
+            # scenario to stop on its own. If it doesn't not a big deal
+            # since it will just be killed instead.
+            reactor.callLater(2, __mark_completed)
 
     def on_ami_connect(self, ami):
         """Callback when AMI connects. Sets test AMI instance."""

-- 
To view, visit https://gerrit.asterisk.org/4923
To unsubscribe, visit https://gerrit.asterisk.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7c95ce6b8d265020cb12dc033ecc9234b522f7e2
Gerrit-PatchSet: 2
Gerrit-Project: testsuite
Gerrit-Branch: master
Gerrit-Owner: Kevin Harwell <kharwell at digium.com>
Gerrit-Reviewer: Anonymous Coward #1000019
Gerrit-Reviewer: George Joseph <gjoseph at digium.com>
Gerrit-Reviewer: Joshua Colp <jcolp at digium.com>



More information about the asterisk-commits mailing list