<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/2041/1/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Asterisk+SCF+Testing">Asterisk SCF Testing</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~dlee">David M. Lee</a>
</h4>
<br/>
<h4>Changes (1)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" >h4. A Note on Regression Testing <br>Regression Testing is a methodology for insuring that defects that are discovered in one version do not reappear in future versions of the software. Regression tests will be implemented via one or more of of the three previously mentioned categories of unit, integration or system tests. It will be up to the project management processes to insure that such tests are created as defects are discovered and addressed, as well as to insure that all tests are run on new versions of the software. <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;"> <br>h4. Details <br> <br>Details on how developers should go about writing unit tests, component tests and integration tests can be found on the [Developer Driven Automated Testing] page. <br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<p>This page is intended to document the approach to testing for the Asterisk SCF Project. Software testing is a huge topic in general, so this page will attempt to focus on specific practices recommended and followed by the Asterisk SCF development team. </p>
<h4><a name="AsteriskSCFTesting-TestingCategories"></a>Testing Categories</h4>
<p>Asterisk SCF will categorize tests into four primary groups: Unit Testing, Component Testing, Integration Testing, and System Testing. </p>
<h6><a name="AsteriskSCFTesting-UnitTesting"></a>Unit Testing </h6>
<p>Unit Testing focuses on a testing specific software classes. Unit Tests provide a mock environment for a specific class to operate in, and exercises the class's interface. The test framework instantiates the class within the test process itself. </p>
<h6><a name="AsteriskSCFTesting-ComponentTesting"></a>Component Testing</h6>
<p>Component Testing focuses on a specific Asterisk SCF component. Asterisk SCF consists of a set of deployable components which expose defined public interfaces to the rest of the system. Any dependent components of the component under test are supplied by the test framework as mock components which provide responses under control of the tests. The test framework uses the public interfaces of the component under test to verify its behavior. </p>
<h6><a name="AsteriskSCFTesting-IntegrationTesting"></a>Integration Testing</h6>
<p>Integration testing verifies that specific components are interoperable with one another. Ideally, pairs of components with strong dependencies are tested, and then larger collections that address specific areas of functionality. </p>
<h6><a name="AsteriskSCFTesting-SystemTesting"></a>System Testing</h6>
<p>This level of testing is designed to validate that the fully integrated system functions as intended and meets certain performance criteria. Since it will be possible to deploy and configure Asterisk SCF in a wide variety of configurations, we will need to identify specific Use Cases of deployment to focus system testing. </p>
<p>System Testing refers to all tests of the fully integrated system, and can be subdivided into more specific categories:</p>
<ul>
<li>Functional Testing</li>
<li>Performance Testing</li>
<li>Load Testing</li>
<li>Stress Testing<br/>
(etc.) </li>
</ul>
<h4><a name="AsteriskSCFTesting-RecommendedTestApproachforAsteriskSCF"></a>Recommended Test Approach for Asterisk SCF</h4>
<p>For each test group, automated tests will be favored wherever possible. Unit Tests and Component Tests, in particular, will all be defined as automated tests. </p>
<h6><a name="AsteriskSCFTesting-AsteriskSCFUnitTestingandComponentTesting"></a>Asterisk SCF Unit Testing and Component Testing</h6>
<p>The Asterisk SCF code base will be developed with automated unit tests and component tests for each component, with the <a href="http://www.boost.org/doc/libs/1_39_0/libs/test/doc/html/index.html" class="external-link" rel="nofollow">Boost Test Library</a> (aka. Boost.Test) used as the test framework. Since the primary development language for Asterisk SCF will be C++, Boost.Test provides a test framework that should be natural for the developers/maintainers of the code. Boost.Test provides a framework that supports:</p>
<ul>
<li>Writing small tests that can be grouped into test suites.</li>
<li>External control over verbosity.</li>
<li>Easily organizing tests in a test tree.</li>
<li>Reusable test fixtures.</li>
<li>Uniform error reporting through the Execution Program Monitor.</li>
</ul>
<p>Unit and Component tests will reside in a parallel structure to the source code in the source code repository, and the build of the test artifacts will be managed by CMake scripts just as the source is. </p>
<h6><a name="AsteriskSCFTesting-AsteriskSCFIntegrationTesting"></a>Asterisk SCF Integration Testing</h6>
<p>Integration testing becomes a bit more complicated. As we attempt to test two or more components, the complexity of providing the components with a defined test environment that allows them to operate just as they will in the actual system can become daunting. Setting up network conditions, configuration files, and other setup tasks required to run a specific test will likely benefit from a flexible, cross-platform scripting capability such as that provided by Python. We expect both Python and Boost.Test to play key roles in integration testing for Asterisk SCF. </p>
<h6><a name="AsteriskSCFTesting-AsteriskSCFSystemTesting"></a>Asterisk SCF System Testing</h6>
<p>System testing for Asterisk SCF will involve deploying Asterisk SCF according to a specific Use Case. For example, Asterisk SCF can be distributed across numerous machines, or deployed on a single machine. For each deployment case, a set of system tests will verify that Asterisk SCF can meet functional and performance design goals. Boost.Test is likely to be ill-suited for managing system-level tests. However, a cross-platform scripting language such as Python, which can also plug into the Ice infrastructure to monitor events, is ideally suited for such testing. </p>
<h4><a name="AsteriskSCFTesting-ANoteonRegressionTesting"></a>A Note on Regression Testing</h4>
<p>Regression Testing is a methodology for insuring that defects that are discovered in one version do not reappear in future versions of the software. Regression tests will be implemented via one or more of of the three previously mentioned categories of unit, integration or system tests. It will be up to the project management processes to insure that such tests are created as defects are discovered and addressed, as well as to insure that all tests are run on new versions of the software. </p>
<h4><a name="AsteriskSCFTesting-Details"></a>Details</h4>
<p>Details on how developers should go about writing unit tests, component tests and integration tests can be found on the <a href="/wiki/display/TOP/Developer+Driven+Automated+Testing" title="Developer Driven Automated Testing">Developer Driven Automated Testing</a> page.</p>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;">
<a href="https://wiki.asterisk.org/wiki/users/viewnotifications.action" class="grey">Change Notification Preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Asterisk+SCF+Testing">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=1736783&revisedVersion=13&originalVersion=12">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Asterisk+SCF+Testing?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>