[asterisk-dev] [Code Review] Add test execution modes to the Asterisk Testsuite

mjordan reviewboard at asterisk.org
Tue Oct 11 11:03:55 CDT 2011


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/1447/
-----------------------------------------------------------

(Updated Oct. 11, 2011, 11:03 a.m.)


Review request for Asterisk Developers and Paul Belanger.


Changes
-------

Removed red blobs


Summary
-------

This patch adds the ability for global 'modes' to be defined for the Asterisk test-suite.  Those settings in a mode can then be applied either at the base level in the runtests.py script, or by any test that inherits from TestClass.py.  As the global settings file makes use of the TestConfig class, all settings in the TestConfig class can theoretically be applied on a global scale - although since each test checks its dependencies independently, only the Test Conditions are currently useful.

One additional settings was added - exclude-tests.  This lets the global settings file explicitly call out tests (or subsets of tests) to not run without having to modify the tests.yaml file, and acts as a counterpart to the --test command line flag.

The only assumption made by this patch is that the global settings file is named 'test-config.yaml' and is located at the current run directory (the location of the runtests.py script).


Diffs (updated)
-----

  /asterisk/trunk/README.txt 2347 
  /asterisk/trunk/lib/python/asterisk/ChannelTestCondition.py PRE-CREATION 
  /asterisk/trunk/lib/python/asterisk/FdTestCondition.py 2358 
  /asterisk/trunk/lib/python/asterisk/TestCase.py 2358 
  /asterisk/trunk/lib/python/asterisk/TestConditions.py 2358 
  /asterisk/trunk/lib/python/asterisk/TestConfig.py 2358 
  /asterisk/trunk/runtests.py 2495 
  /asterisk/trunk/test-config.yaml PRE-CREATION 
  /asterisk/trunk/tests/apps/voicemail/authenticate_extensions/run-test 2394 
  /asterisk/trunk/tests/apps/voicemail/authenticate_extensions/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/authenticate_invalid_mailbox/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/authenticate_invalid_password/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/authenticate_nominal/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_callback/configs/ast1/extensions.conf 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_callback/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_delete/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_dialout/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_envelope/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_hangup/configs/ast2/extensions.conf 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_hangup/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_with_prepend/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user_hangup/configs/ast2/extensions.conf 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user_hangup/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_nominal/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_change_password/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_busy/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_name/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_temp/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_unavail/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_reply/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_while_leaving_msg/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/func_vmcount/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_contexts/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_external_notification/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_forwarding/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_forwarding_auto_urgent/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_nominal/test-config.yaml 2394 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_priority/test-config.yaml 2394 

Diff: https://reviewboard.asterisk.org/r/1447/diff


Testing
-------

Tested fast config mode, standard config mode, and pessimistic config mode.  Each worked as intended:
* Fast config mode successfully excluded those tests with a reactor_timeout greater than one minute
* Standard config mode applied no global settings
* Pessimistic config mode applied the Test Conditions to all tests inheriting from TestCase.  Those tests that further defined test specific overrides applied those settings over the global conditions.


Thanks,

mjordan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20111011/c59286e0/attachment.htm>


More information about the asterisk-dev mailing list