[asterisk-dev] [Code Review] 4080: Test Suite: Fix the 'expected-result' YAML property for test configuration

[Scott Griepentrog]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/4080/#review13590
-----------------------------------------------------------

Ship it!


I tested all combinations of pass/fail and it works correctly.

- Scott Griepentrog


On Oct. 21, 2014, 9:57 a.m., jbigelow wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/4080/
> -----------------------------------------------------------
> 
> (Updated Oct. 21, 2014, 9:57 a.m.)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Repository: testsuite
> 
> 
> Description
> -------
> 
> When the 'expected-result' (or 'expectedResult') YAML property for test configuration is set to False and the test fails, the test should be marked as passed. However it is marked as failed. This patch should fix the issue so that tests are marked as passed in this scenario.
> 
> Additionally:
> * Check if p.returncode is not zero so self.passed is a boolean rather than an int in some cases.
> * Added some print statements to make it clear why a test was marked as passed or failed when the 'expected-result' YAML property is set to False.
> * Added text to the failure message so it's easily known when looking at the results file that the test was expected to fail but passed and therefore marked as failed.
> 
> 
> Diffs
> -----
> 
>   /asterisk/trunk/runtests.py 5766 
>   /asterisk/trunk/lib/python/asterisk/test_config.py 5766 
> 
> Diff: https://reviewboard.asterisk.org/r/4080/diff/
> 
> 
> Testing
> -------
> 
> Tested the various scenarios and they all seem to properly work as expected now.
> 
> 
> Thanks,
> 
> jbigelow
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20141022/ede908f7/attachment-0001.html>