<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/en/2172/18/9/_/styles/combined.css?spaceKey=AST&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/AST/Overview">Overview</a></h2>
<h4>Page <b>added</b> by <a href="https://wiki.asterisk.org/wiki/display/~mjordan">Matt Jordan</a>
</h4>
<br/>
<div class="notificationGreySide">
<h1><a name="Overview-Motivation"></a>Motivation</h1>
<p>At its heart, the Asterisk Test Suite is a way of executing programs that test functionality in Asterisk. Historically, the Test Suite has implemented these tests in a variety of languages - Lua, bash, Python - but there is no hard rule that limits what is supported. If your implementation supports having an executable file named <em>runtests</em>, then it is acceptable by the Test Suite.</p>
<p>There are some problems that arise with this level of flexibility. Tests often have to do many of the same tasks: start and stop Asterisk, connect over AMI and originate calls, run AGI scripts, gather and report results, etc. For each language chosen, sets of libraries have grown to support these common activities. This has resulted in duplication of functionality between libraries written in different languages. Over time, some libraries have also received more attention than others as well, resulting in functionality that is only available in a particular language.</p>
<p>As a result, we've started to focus on writing our tests in Python. There is no strict enforcement of this, but since we have developed a number of libraries to support test execution - and continue to focus our efforts on expanding and enhancing those libraries - you may get more "bang for your buck" by writing tests in Python as well.</p>
<p>All of that said, even when a single implementation language is chosen, there is still a fair amount of repeated code throughout the Test Suite. Because tests support their own execution, they often have to implement similar mechanisms. Each test must have its own execution entry point, instantiate its own objects, do the minimum amount needed to get Asterisk running, etc. Between tests that cover similar functionality, there is even more repeated code. Moving shared code into libraries helps alleviate that to some extent, but even then some amount of duplication occurs.</p>
<h1><a name="Overview-AnExampletwoTraditionalTestsusingSIPp"></a>An Example - two Traditional Tests using SIPp</h1>
<p>As an example, consider two tests written in Python that run SIPp scenarios - <b>message_auth</b> and <b>message_disabled</b>.</p>
<h5><a name="Overview-messageauth"></a>message_auth</h5>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">#!/usr/bin/env python
'''
Copyright (C) 2010, Digium, Inc.
Russell Bryant <russell@digium.com>
This program is free software, distributed under the terms of
the GNU General Public License Version 2.
'''
import sys
import os
sys.path.append("lib/python")
from twisted.internet import reactor
from asterisk.sipp import SIPpTest
WORKING_DIR = "sip/message_auth"
TEST_DIR = os.path.dirname(os.path.realpath(__file__))
SIPP_SCENARIOS = [
{
'scenario' : 'message_recv.xml',
'-p' : '5062'
},
{
'scenario' : 'message.xml',
'-p' : '5061'
}
]
def main():
test = SIPpTest(WORKING_DIR, TEST_DIR, SIPP_SCENARIOS)
reactor.run()
if not test.passed:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())</pre>
</div></div>
<h5><a name="Overview-messagedisabled"></a>message_disabled</h5>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">#!/usr/bin/env python
'''
Copyright (C) 2010, Digium, Inc.
Russell Bryant <russell@digium.com>
This program is free software, distributed under the terms of
the GNU General Public License Version 2.
'''
import sys
import os
sys.path.append("lib/python")
from twisted.internet import reactor
from asterisk.sipp import SIPpTest
WORKING_DIR = "sip/message_disabled"
TEST_DIR = os.path.dirname(os.path.realpath(__file__))
SIPP_SCENARIOS = [
{
'scenario' : 'message.xml',
}
]
def main():
test = SIPpTest(WORKING_DIR, TEST_DIR, SIPP_SCENARIOS)
reactor.run()
if not test.passed:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())</pre>
</div></div>
<p>Both of these are very straight-forward tests that simply execute their SIPp scenarios and base their pass/fail status on the result of those scenarios. Even still, there is repeated code:</p>
<ul>
<li>Duplicate entry points for execution</li>
<li>Manipulation of the <a href="http://twistedmatrix.com/trac/" class="external-link" rel="nofollow">twisted</a> reactor</li>
<li>Instantiation of the test object, and reading of its result</li>
</ul>
<p>In fact, if we look at what is unique, its consists only of the test and working directories (which are easily determined by higher level entities), and the SIPp scenarios to run. If the only differences between tests can be expressed by configuration, why not do that?</p>
<h1><a name="Overview-APluggableFrameworkImplementation"></a>A Pluggable Framework Implementation</h1>
<p>These observations led to the development of a pluggable component framework for the Asterisk Test Suite. Tests that support this framework specify what components they need and their configuration in their <em>test-config.yaml</em> configuration files. A Python module, <em>TestRunner.py</em>, is responsible for starting execution, instantiating the components and injecting their dependencies, starting the test, and reporting results. The actual "business logic" of the test itself is deferred to the components specified in the test configuration.</p>
<p>Looking again at the two tests using SIPp, their <em>test-config.yaml</em> could express those tests in the following manner:</p>
<h5><a name="Overview-messageauth"></a>message_auth</h5>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">testinfo:
summary: 'Test response to MESSAGE when out-of-call MESSAGE is disabled (or not supported)'
description: |
'Send Asterisk a MESSAGE and expect that it gets denied'
test-modules:
test-object:
config-section: test-object-config
typename: 'SIPpTest.SIPpTest'
test-object-config:
pass-on-sipp-scenario: True
scenarios:
-
scenario: 'message_recv.xml'
-p: '5062'
-
scenario: 'message.xml'
-p: '5061'</pre>
</div></div>
<h5><a name="Overview-messagedisabled"></a>message_disabled</h5>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">testinfo:
summary: 'Test response to MESSAGE when out-of-call MESSAGE is disabled (or not supported)'
description: |
'Send Asterisk a MESSAGE and expect that it gets denied'
test-modules:
test-object:
config-section: test-object-config
typename: 'SIPpTest.SIPpTest'
test-object-config:
pass-on-sipp-scenario: True
scenarios:
-
scenario: 'message.xml'
properties:
minversion: '1.8.0.0'
dependencies:
- app : 'sipp'
tags:
- SIP</pre>
</div></div>
<p>All of this, hopefully, gives the following benefits:</p>
<ul>
<li>Less time to write a test, particularly if you have a large number of tests that cover related functionality, or your tests can leverage existing Python libraries</li>
<li>Some more difficult concepts - such as asynchronous event programming using twisted - may be deferred to lower level libraries (bad pun)</li>
<li>Quicker execution time, by taking advantage of byte code files compiled by the Python interpreter</li>
<li>Shared functionality across tests</li>
<li>Common implementation of similar functionality across tests, easing test maintenance</li>
</ul>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;" class="grey">
<a href="https://wiki.asterisk.org/wiki/users/removespacenotification.action?spaceKey=AST">Stop watching space</a>
<span style="padding: 0px 5px;">|</span>
<a href="https://wiki.asterisk.org/wiki/users/editmyemailsettings.action">Change email notification preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/AST/Overview">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/display/AST/Overview?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>