[asterisk-users] Testing Framework

[dave cantera]

matt,
are you looking for unit testing of the * components or systems testing, 
testing the finished product?  or both?
I think you are onto something here...  I hope it takes root.  I would 
say put it in the addons.  it would be Great if digium takes it up. it 
is a smart move for them to foster, cajole, nudge, and support it. 
call volume I would leave to others as different processors, O/S, 
builds, kernel versions, and configurations will have too many variables.

I was playing with the idea of monitoring multiple * systems.  perhaps 
we can start out with testing the components and then migrate the 
project (future) to one pbx monitor the other.  we will need scripts to 
initiate some action, config to make some measurements, the scripts to 
gather the results into a nice neat little summary report.  you will 
want to take the human aspect out of the picture as much as possible.  
for example:

    on pbx A

        * create a recording in multiple formats .gsm, .wav, etc.
        * initiate a script to generate 5,10, or 25 calls to pbx B and
          play the file

    on pbx B

        * pbx B gets the calls, records them,
        * copy the recordings from pbx A to pbx B (or have that already
          done)
        * have a wave analyzer compare the recordings to the original
          files (you know I won't be writing that program! :)
        * report on anomalies

*call
* 	*Technology
* 	*recording
delta
*
1
	Zap Provider 1
	2%
2
	VoIP Provider 2
	5%
3
	VoIP Provider 2
	15%
...
	VoIP Provider 3
	...


let me know what you think!
daveC



Matt Riddell wrote:
> Hash: SHA1
>
> Hi,
>
> So, now that we've all complained about the state of testing of Open
> Source versions of Asterisk, lets do something about it.
>
> I propose we start with a list of things that we think should be tested
> in Asterisk, and means to test them.
>
> Maybe we could run certain tests based on the changes between minor
> versions?
>
> Anyway lets start.
>
> Call Volumes
>
> 1) Call volume up to x channels from SIP to SIP (i.e. sipp)
> 2) Call volume up to x channels from IAX2 to SIP
> 3) Call volume up to x channels from IAX2 to IAX2
>
> Application testing
>
> 4) Connect x calls between techs to Meetme (leave running for 1 hour)
> 5) Connect x concurrent calls to VoiceMail
>
> Call Centre Testing
>
> 6) Send x calls to a queue with no agents in it, leave them holding for
> x minutes
> 7) Run x calls against AMD connected to recorded known good files
>
> Recording
>
> 8) Run x calls recording simultaneously from an automatically generated
> call, play ulaw/alaw - compare outputs.
>
> You get the idea.
>
> If people can add to this list, I can start making a few scripts and
> programs that will test them (as I'm sure others can).
>
> If we end up with a complete list, I'm sure some of our individual QA
> departments can take the responsibility for certain items.
>
> The call volume ones are obviously going to either need a live person to
> dial in at volume and check everything is ok, or a recording which can
> later be checked.
>
> I'm of the opinion that the majority of tests should test individual
> components, but that we should also form some "Application Type"
> frameworks so that we can test integration between Asterisk apps.
>
> Any takers?  Add to the list?  If there is something you believe is
> mission critical to your business, write up a test case for it, and
> we'll all try to code something that can run automatically to test it.
>
> If we try and keep to ANSI C for the testing apps, Digium should be able
> to run them on their multi platform machines as well.
>
> Should these tests be added to Asterisk-Addons or maintained outside of
> the tree?
>
> Anyway, what do you think? Feasible? I already have a few tests here and
> I'm sure others have a few too.  Lets put them all together and get a
> framework going.
>
> - --
> Kind Regards,
>
> Matt Riddell
> Director
> _______________________________________________
>
> http://www.venturevoip.com (Great new VoIP end to end solution)
> http://www.venturevoip.com/news.php (Daily Asterisk News - html)
> http://feeds.venturevoip.com/AsteriskNews (Daily Asterisk News - rss)
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFG1yKhDQNt8rg0Kp4RAv5UAJ48tW28T5lWCQIPTwVimyvlhEPJowCgpnE6
> OF3L2M/6Hc+YBNL1NFx6dzA=
> =OXNn
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
>
>
>
>   

-- 
My wife's sister is in California.  
I should buy her a Videophone2008!

Truly, The Next Best Thing to Being There!
--

WorldWideVideoPhones.com
856.380.0894