[asterisk-dev] Bamboo incorrectly reporting 26 tests passed, when nothing was actually tested

[Russell Bryant]

On 03/04/2010 03:44 AM, Alec Davis wrote:
> but
>   <http://bamboo.asterisk.org/browse/AST-TRUNK-151/log>
> http://bamboo.asterisk.org/browse/AST-TRUNK-151/log show no tests were
> actually run, but<http://bamboo.asterisk.org/browse/AST-TRUNK-151/log>
> http://bamboo.asterisk.org/browse/AST-TRUNK-151 reports 27 succesful
> tests????
> 03-Mar-2010 15:40:20  Running all available tests...
> 03-Mar-2010 15:40:20
> 03-Mar-2010 15:40:20  START  apps/app_voicemail/ - test_voicemail_msgcount
> 03-Mar-2010 15:40:20  [app_voicemail.c:test_voicemail_msgcount:11781]:
> Unable to clear test directory: unable to fork()
> 03-Mar-2010 15:40:20  END    apps/app_voicemail/ - test_voicemail_msgcount
> Time: 17ms Result: NOT RUN
> 03-Mar-2010 15:40:20  START  apps/app_voicemail/ - vmsayname_exec
> 03-Mar-2010 15:40:20  [app_voicemail.c:test_voicemail_vmsayname:11693]: Test
> playing of extension when greeting is not available...
> 03-Mar-2010 15:40:25  *** Generating Unit Test Results Output ***
> 03-Mar-2010 15:40:31  TEST_RESULTS_DIR:
> /srv/bamboo/xml-data/build-dir/AST-TRUNK/test-reports
> 03-Mar-2010 15:40:31  total 4
> 03-Mar-2010 15:40:31  -rw-r--r-- 1 root root 1892 Mar  3 15:40
> unit-test-results.xml
> 03-Mar-2010 15:40:31  Test Results:

What you're seeing here is that all of the output from the asterisk -rx 
command to run the tests is not making it back to the log.  I'm sure it 
has something to do with the vmsayname_exec test taking over 7 seconds 
to run.

However, all of the tests do run and the results are still captured. 
It's not possible for the test results to be stale, as every build and 
test cycle is done in a fresh checkout.

-- 
Russell Bryant
Digium, Inc. | Engineering Manager, Open Source Software
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
www.digium.com -=- www.asterisk.org -=- blogs.asterisk.org