[asterisk-dev] [Code Review] 4080: Test Suite: Fix the 'expected-result' YAML property for test configuration

[jbigelow]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/4080/
-----------------------------------------------------------

(Updated Oct. 15, 2014, 5:34 p.m.)


Review request for Asterisk Developers.


Changes
-------

Removed 'NOTICE: ' from strings that are printed.


Repository: testsuite


Description
-------

When the 'expected-result' (or 'expectedResult') YAML property for test configuration is set to False and the test fails, the test should be marked as passed. However it is marked as failed. This patch should fix the issue so that tests are marked as passed in this scenario.

Additionally:
* Check if p.returncode is not zero so self.passed is a boolean rather than an int in some cases.
* Added some print statements to make it clear why a test was marked as passed or failed when the 'expected-result' YAML property is set to False.
* Added text to the failure message so it's easily known when looking at the results file that the test was expected to fail but passed and therefore marked as failed.


Diffs (updated)
-----

  /asterisk/trunk/runtests.py 5726 
  /asterisk/trunk/lib/python/asterisk/test_config.py 5726 

Diff: https://reviewboard.asterisk.org/r/4080/diff/


Testing
-------

Tested the various scenarios and they all seem to properly work as expected now.


Thanks,

jbigelow

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20141015/ed395d97/attachment.html>