[Asterisk-code-review] subscriptions/rls: Race conditions in some of the batched tests (testsuite[master])

[George Joseph]

Hello Anonymous Coward #1000019, Joshua Colp,

I'd like you to reexamine a change.  Please visit

    https://gerrit.asterisk.org/4923

to look at the new patch set (#2).

Change subject: subscriptions/rls: Race conditions in some of the batched tests
......................................................................

subscriptions/rls: Race conditions in some of the batched tests

Several of the rls list and list of lists mwi and presence batched tests could
fail due to a race between the rls_test and driver pluggable modules. Most of
the patched tests had multiple modules that could stop the running test and
depending on which one stop the test first some of the test conditions may not
have been checked yet.

First, all tests were initiated by the sipp test module. If the 'stop-after-
scenarios' option was not set to false then once the sipp scenario ended it
would stop the test. This patch sets the option to false on the offending
tests. It also removes any pause statements from the sipp scenario that were
there only to keep the scenario from ending and stopping the test too soon.

Second, the rls_test pluggable module would also stop the test once all its
configured conditions had been met. It too contains an option, 'stop-test-
after-notifys', that when set to false will not stop the test after validating
conditions. However, for several of the tests this option was not set and as it
defaults to true the test would sometimes end early. This patch sets that option
to false for the tests that needed it.

Third, all the problematic tests have/had a driver pluggable module that also
check against certain conditions and then would either stop the test too, or
would wait until one of the other modules mentioned above would. Again, though
if one of the ones above stopped the test before meeting the conditions of this
module then the test would fail. Or in the cases where this module stops the
test first the others may not have finished.

In order to alleviate any race conditions and also make tests deterministic,
all tests were given one stopping point on success. Some of the tests just
needed the options mentioned above set appropriately. Others required the
driver module to be removed and their conditions are instead checked by using
the EventActionModule. For these an UserEvent is now raised once rls_test is
done. The EventActionModule can then wait for that event and the other ami
conditional events before stopping.

Change-Id: I7c95ce6b8d265020cb12dc033ecc9234b522f7e2
---
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/basic/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/multiple_resources_single_change/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/resubscription_interruption/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/single_resource_multiple_changes/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/mwi/batched/termination_interruption/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/basic/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/multiple_resources_single_change/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/resubscription_interruption/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/single_resource_multiple_changes/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/driver.py
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/sipp/subscribe.xml
M tests/channels/pjsip/subscriptions/rls/lists/nominal/presence/batched/termination_interruption/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/driver.py
M tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/mwi/batched/test-config.yaml
D tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/driver.py
M tests/channels/pjsip/subscriptions/rls/lists_of_lists/nominal/presence/batched/test-config.yaml
M tests/channels/pjsip/subscriptions/rls/rls_test.py
23 files changed, 371 insertions(+), 610 deletions(-)


  git pull ssh://gerrit.asterisk.org:29418/testsuite refs/changes/23/4923/2
-- 
To view, visit https://gerrit.asterisk.org/4923
To unsubscribe, visit https://gerrit.asterisk.org/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7c95ce6b8d265020cb12dc033ecc9234b522f7e2
Gerrit-PatchSet: 2
Gerrit-Project: testsuite
Gerrit-Branch: master
Gerrit-Owner: Kevin Harwell <kharwell at digium.com>
Gerrit-Reviewer: Anonymous Coward #1000019
Gerrit-Reviewer: Joshua Colp <jcolp at digium.com>