[asterisk-users] Automatic tests (was Re: Upgrade to Asterisk 1.4 - it's one year's old!)

[Atis Lezdins]

On 12/17/07, Jared Smith <jsmith at digium.com> wrote:
> On Mon, 2007-12-17 at 12:00 -0800, shadowym wrote:
> > I do wish Digium or whoever tests this stuff had a more reliable way of
> > testing software releases rather than relying on feedback from the
> > community.  Fonality, for example use what they call a "hammer" which sounds
> > to me like a bunch of servers running various stress tests on the software
> > to try break it.
>
> This "hammer" of which you speak is a commercial program from Empririx,
> part of their Hammer line of VoIP testing products.[1]  Just to be fair
> and honest, Digium has a copy of the Empirix Hammer software and uses it
> to test Asterisk.  They also spend countless hours testing Asterisk in
> other ways as well.  Part of the problem of testing comes from high
> number of combinations of different components that must be tested.
> Just testing calls between the three most common channel drivers (SIP,
> IAX2, and Zap) involves nine tests at a minimum:
>
> SIP->SIP
> SIP->Zap
> SIP->IAX2
> IAX2->IAX2
> IAX2->SIP
> IAX2->Zap
> Zap->Zap
> Zap->SIP
> Zap->IAX2
>
> Obviously, within each of those tests, there are lots of different
> options that could be tested as well (such as methods for sending
> DTMF).
>
> I've offered to start pulling together a community-driven set of tests
> that we can automate and run against Asterisk on a regular basis, but so
> far nobody has offered up any help in this regard, and I've been busy
> with other things (like teaching Asterisk training classes) that I
> haven't had any time to devote to it myself.  I'm hoping to be able to
> start working on a testing framework sometime in January, as long as I
> don't get too many other things put on my plate between now and then.

Have you seen testing framework we created? I posted it some time ago,
but unfortuneately i didn't get much feedback. It might be not so
intuitive to configure, but it really helped us to find memory
corruption problems under high load.

Please see:
http://lists.digium.com/pipermail/asterisk-users/2007-November/200429.html

Regards,
Atis

-- 
Atis Lezdins
VoIP Developer,
IQ Labs Inc.
atis at iq-labs.net
Skype: atis.lezdins
Cell Phone: +371 28806004
Work phone: +1 800 7502835