[svn-commits] jbigelow: testsuite/asterisk/trunk r5767 - in /asterisk/trunk: ./ lib/python/...

[SVN commits to the Digium repositories]

Author: jbigelow
Date: Fri Oct 24 08:38:58 2014
New Revision: 5767

URL: http://svnview.digium.com/svn/testsuite?view=rev&rev=5767
Log:
Fix 'expected-result' YAML property & santize return code from test_runner.py

This fixes the 'expected-result' YAML property for test configurations. When
it is set to False and the test fails, the test will properly be marked as
passed. When is is set to False and the test passes, the test will properly be
marked as failed since it didn't match what was expected.

Additionally this sanitizes the return code from test_runner.py so that
'self.pass' is always a boolean whereas before there were some scenarios that
it would be an int.

Review: http://reviewboard.digium.internal/r/899/


Modified:
    asterisk/trunk/lib/python/asterisk/test_config.py
    asterisk/trunk/runtests.py

Modified: asterisk/trunk/lib/python/asterisk/test_config.py
URL: http://svnview.digium.com/svn/testsuite/asterisk/trunk/lib/python/asterisk/test_config.py?view=diff&rev=5767&r1=5766&r2=5767
==============================================================================
--- asterisk/trunk/lib/python/asterisk/test_config.py (original)
+++ asterisk/trunk/lib/python/asterisk/test_config.py Fri Oct 24 08:38:58 2014
@@ -320,9 +320,9 @@
                 self.feature_check[self.minversion.feature] = False
         if "maxversion" in properties:
             self.maxversion = AsteriskVersion(properties["maxversion"])
-        self.expect_pass = (properties.get("expectedResult") or
-                            properties.get("expected-result") or
-                            True)
+        self.expect_pass = (
+                properties.get("expectedResult", self.expect_pass) and
+                properties.get("expected-result", self.expect_pass))
         if "tags" in properties:
             self.tags = properties["tags"]
         if "features" in properties:

Modified: asterisk/trunk/runtests.py
URL: http://svnview.digium.com/svn/testsuite/asterisk/trunk/runtests.py?view=diff&rev=5767&r1=5766&r2=5767
==============================================================================
--- asterisk/trunk/runtests.py (original)
+++ asterisk/trunk/runtests.py Fri Oct 24 08:38:58 2014
@@ -69,9 +69,19 @@
                 pass
             p.wait()
 
+            # Sanitize p.returncode so it's always a boolean.
+            did_pass = (p.returncode == 0)
+            if did_pass and not self.test_config.expect_pass:
+                msg = "Test passed but was expected to fail."
+                print msg
+                self.stdout += msg + "\n"
+            if not did_pass and not self.test_config.expect_pass:
+                print "Test failed as expected."
+
             self.__parse_run_output(self.stdout)
 
-            self.passed = (p.returncode == 0 and self.test_config.expect_pass) or (p.returncode and not self.test_config.expect_pass)
+            self.passed = (did_pass == self.test_config.expect_pass)
+
             core_dumps = self._check_for_core()
             if (len(core_dumps)):
                 print "Core dumps detected; failing test"