[Asterisk-code-review] runtests.py: Add parameter --randomorder. (testsuite[master])

[Corey Farrell]

Corey Farrell has posted comments on this change.

Change subject: runtests.py: Add parameter --randomorder.
......................................................................


Patch Set 1:

> I don't see much usefulness for this feature.  Making the order
 > random is just going to give you a variable wait time till it hits
 > a failing test.

The testsuite takes 8.5 hours to run on my system.  If I make changes that cause many failures in tests/rest_api, it is about 7 hours before the first test in that folder is run.  In random order, on average every 8th test will be against that folder, so it is likely to run a couple tests in the first 10 minutes.  I'm picking on rest_api because it is a lot of tests and they run last.

The idea is that after 30 minutes the testsuite will hit most major parts of Asterisk, I can decide then if I need to cancel the run to make fixes now, or let it continue and check tomorrow.  I am not proposing a single letter option for this.  It is useful to me, but I realize that many people have faster computers so they wouldn't care.

-- 
To view, visit https://gerrit.asterisk.org/550
To unsubscribe, visit https://gerrit.asterisk.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I002c6fa2c7a05e44673cb56b11d4e4c381f3eba0
Gerrit-PatchSet: 1
Gerrit-Project: testsuite
Gerrit-Branch: master
Gerrit-Owner: Corey Farrell <git at cfware.com>
Gerrit-Reviewer: Corey Farrell <git at cfware.com>
Gerrit-Reviewer: Richard Mudgett <rmudgett at digium.com>
Gerrit-HasComments: No