[asterisk-dev] [Code Review] New TestEvent AMI event

[Paul Belanger]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/1364/#review4064
-----------------------------------------------------------


I haven't tested this yet, however this look exciting!  You've nailed one of the main issues with the testsuite; timing.  So obvious questions come to mind about defining 'states' and documenting them properly, and then making sure the testsuite keeps in sync with them.

Also, we might not be able to merge this into 1.8, since this is a new feature, however we did backport some testing functionality into 1.4 previously.

I'll schedule some time this week to give this a run, and hopefully have more questions / comments then.


/branches/1.8/apps/app_voicemail.c
<https://reviewboard.asterisk.org/r/1364/#comment8015>

    Assuming we decide to take this path, wouldn't it make sense just to wrap ast_test_suite_event_notify('PLAYBACK', ...) when ever Asterisk played back an audio file?


- Paul


On Aug. 15, 2011, 4:11 p.m., mjordan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/1364/
> -----------------------------------------------------------
> 
> (Updated Aug. 15, 2011, 4:11 p.m.)
> 
> 
> Review request for Asterisk Developers, David Vossel and Paul Belanger.
> 
> 
> Summary
> -------
> 
> This review is part 1 of 2.
> 
> Previously, when attempting to write tests for complex applications, tests run into two problems:
> 
> 1. The tests become inherently timing driven.  For example, in an application that has a built in playback of a voice file, the test must use Wait(n) or some other sleep mechanism and attempt to wait until the voice prompt is finished before taking action.  This can be even more difficult in those applications that have built in menus whose prompts can change based on the state of the application, such as app_voicemail's VoiceMailMain application.
> 
> 2. Applications whose success is a hangup result can be difficult to verify.  While UserEvent can be used in a dialplan to check certain return paths (thus determining whether an application definitely failed), it becomes difficult to say whether or not an application succeeds.  For example, Dial has multiple success paths where the caller is hung up on; however, the hang up could also be an error in the dialplan.  The only verification mechanism is by inspection.
> 
> The TestEvent manager event attempts to solve these two problems.  It allows a developer to raise a manager event within an application that notifies the Asterisk Test Suite that something has happened in the application.  Currently, this is one of two notifications:
> 1. A generic change in application state
> 2. A failure in an assert, indicating an off nominal condition
> 
> Both of these statements are macros that evaluate to nothing if the TEST_FRAMEWORK compilation flag is not defined, meaning they should have no impact on non-developer compiled code.  While placing the manager events in the code perturbs the units under test, the methods are fairly small, and, in the context of sending audio over a channel for several seconds, are relatively small in terms of timing.
> 
> Part 2 of this review (1363) demonstrates a test in the Asterisk Test Suite using this new event.  This diff also includes some initial hooks using the TestEvent in app_voicemail's VoiceMailMain application - I expect to add additional hooks as further tests are developed (the current ones check some of the nominal paths and operations)
> 
> 
> Diffs
> -----
> 
>   /branches/1.8/include/asterisk/manager.h 331633 
>   /branches/1.8/include/asterisk/test.h 331633 
>   /branches/1.8/main/manager.c 331633 
>   /branches/1.8/main/test.c 331633 
>   /branches/1.8/apps/app_voicemail.c 331633 
> 
> Diff: https://reviewboard.asterisk.org/r/1364/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> mjordan
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20110815/edaad399/attachment-0001.htm>