[asterisk-dev] [Code Review] 4080: Test Suite: Fix the 'expected-result' YAML property for test configuration

[Scott Griepentrog]

> On Oct. 16, 2014, 5:29 p.m., Scott Griepentrog wrote:
> > /asterisk/trunk/runtests.py, line 75
> > <https://reviewboard.asterisk.org/r/4080/diff/2/?file=68354#file68354line75>
> >
> >     This should include a + "\n" like line 59 does.
> 
> jbigelow wrote:
>     Line 59 adds a newline to visually separate the test that was run from the tests output. Line 67 doesn't add a new line to the tests output and I don't believe I should just for this message that I'm appending to the output either.

My concern is that another function executing later could (optionally, on some error) write to self.stdout.  If there is zero possibility of that, then omitting the \n is not an issue.


- Scott


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/4080/#review13558
-----------------------------------------------------------


On Oct. 20, 2014, 11:38 a.m., jbigelow wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/4080/
> -----------------------------------------------------------
> 
> (Updated Oct. 20, 2014, 11:38 a.m.)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Repository: testsuite
> 
> 
> Description
> -------
> 
> When the 'expected-result' (or 'expectedResult') YAML property for test configuration is set to False and the test fails, the test should be marked as passed. However it is marked as failed. This patch should fix the issue so that tests are marked as passed in this scenario.
> 
> Additionally:
> * Check if p.returncode is not zero so self.passed is a boolean rather than an int in some cases.
> * Added some print statements to make it clear why a test was marked as passed or failed when the 'expected-result' YAML property is set to False.
> * Added text to the failure message so it's easily known when looking at the results file that the test was expected to fail but passed and therefore marked as failed.
> 
> 
> Diffs
> -----
> 
>   /asterisk/trunk/runtests.py 5726 
>   /asterisk/trunk/lib/python/asterisk/test_config.py 5726 
> 
> Diff: https://reviewboard.asterisk.org/r/4080/diff/
> 
> 
> Testing
> -------
> 
> Tested the various scenarios and they all seem to properly work as expected now.
> 
> 
> Thanks,
> 
> jbigelow
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20141021/52cd21fa/attachment-0001.html>