[asterisk-users] Testing Framework

[Matt Riddell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

So, now that we've all complained about the state of testing of Open
Source versions of Asterisk, lets do something about it.

I propose we start with a list of things that we think should be tested
in Asterisk, and means to test them.

Maybe we could run certain tests based on the changes between minor
versions?

Anyway lets start.

Call Volumes

1) Call volume up to x channels from SIP to SIP (i.e. sipp)
2) Call volume up to x channels from IAX2 to SIP
3) Call volume up to x channels from IAX2 to IAX2

Application testing

4) Connect x calls between techs to Meetme (leave running for 1 hour)
5) Connect x concurrent calls to VoiceMail

Call Centre Testing

6) Send x calls to a queue with no agents in it, leave them holding for
x minutes
7) Run x calls against AMD connected to recorded known good files

Recording

8) Run x calls recording simultaneously from an automatically generated
call, play ulaw/alaw - compare outputs.

You get the idea.

If people can add to this list, I can start making a few scripts and
programs that will test them (as I'm sure others can).

If we end up with a complete list, I'm sure some of our individual QA
departments can take the responsibility for certain items.

The call volume ones are obviously going to either need a live person to
dial in at volume and check everything is ok, or a recording which can
later be checked.

I'm of the opinion that the majority of tests should test individual
components, but that we should also form some "Application Type"
frameworks so that we can test integration between Asterisk apps.

Any takers?  Add to the list?  If there is something you believe is
mission critical to your business, write up a test case for it, and
we'll all try to code something that can run automatically to test it.

If we try and keep to ANSI C for the testing apps, Digium should be able
to run them on their multi platform machines as well.

Should these tests be added to Asterisk-Addons or maintained outside of
the tree?

Anyway, what do you think? Feasible? I already have a few tests here and
I'm sure others have a few too.  Lets put them all together and get a
framework going.

- --
Kind Regards,

Matt Riddell
Director
_______________________________________________

http://www.venturevoip.com (Great new VoIP end to end solution)
http://www.venturevoip.com/news.php (Daily Asterisk News - html)
http://feeds.venturevoip.com/AsteriskNews (Daily Asterisk News - rss)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG1yKhDQNt8rg0Kp4RAv5UAJ48tW28T5lWCQIPTwVimyvlhEPJowCgpnE6
OF3L2M/6Hc+YBNL1NFx6dzA=
=OXNn
-----END PGP SIGNATURE-----