<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/2041/1/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Testing">Testing</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~dsessions@digium.com">Darren Sessions</a>
</h4>
<br/>
<h4>Changes (3)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{gliffy:name=flow chart test env|align=left|size=L|version=1} <br></td></tr>
<tr><td class="diff-unchanged" >{note}This is a work in progress.{note} <br>h2. Overview <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" >Automated build and unit testing is done when commits to the source-code are made. Additionally, an exhaustive set of tests covering component, integration, and system testing is done on a nightly basis using an automated virtualization environment. This nightly-testing produces data from the various tools that is then collected, checked, and then those results along with network capture files, call flows, etc. are all fed back into the Bamboo job where they are then saved as artifacts for reporting purposes, issue triage, and for possible use in subsequent stages within the test run. <br> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{gliffy:name=test env build and unit tests flow chart|align=center|size=L|version=3} <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">{gliffy:name=flow chart test env|align=center|size=L|version=1} <br></td></tr>
<tr><td class="diff-unchanged" > <br>h2. 3rd Party Tools <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/wiki/images/icons/emoticons/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td>This is a work in progress.</td></tr></table></div>
<h2><a name="Testing-Overview"></a>Overview</h2>
<p>Tests are categorized into five primary groups: Build Testing, Unit Testing, Component Testing, Integration Testing, and System Testing. </p>
<h6><a name="Testing-BuildTesting"></a>Build Testing</h6>
<p>Build Testing verifies that the Asterisk SCF source-code builds properly on a broad spectrum of operating systems, distributions, and architectures.</p>
<h6><a name="Testing-UnitTesting"></a>Unit Testing </h6>
<p>Unit Testing focuses on a testing specific software classes. Unit Tests provide a mock environment for a specific class to operate in, and exercises the class's interface. The test framework instantiates the class within the test process itself. </p>
<h6><a name="Testing-ComponentTesting"></a>Component Testing</h6>
<p>Component Testing focuses on a specific Asterisk SCF component. Asterisk SCF consists of a set of deployable components which expose defined public interfaces to the rest of the system. Any dependent components of the component under test are supplied by the test framework as mock components which provide responses under control of the tests. The test framework uses the public interfaces of the component under test to verify its behavior. </p>
<h6><a name="Testing-IntegrationTesting"></a>Integration Testing</h6>
<p>Integration testing verifies that specific components are interoperable with one another. Ideally, pairs of components with strong dependencies are tested, and then larger collections that address specific areas of functionality. </p>
<h6><a name="Testing-SystemTesting"></a>System Testing</h6>
<p>This level of testing is designed to validate that the fully integrated system functions as intended and meets certain performance criteria. Since it will be possible to deploy and configure Asterisk SCF in a wide variety of configurations, we will need to identify specific Use Cases of deployment to focus system testing. </p>
<p>System Testing refers to all tests of the fully integrated system, and can be subdivided into more specific categories:</p>
<ul>
<li>Functional Testing</li>
<li>Performance Testing (Optimizations turned on)</li>
<li>Load Testing</li>
<li>Stress Testing</li>
<li>IPv4 / IPv6<br/>
(etc.) </li>
</ul>
<h2><a name="Testing-TestingEnvironment"></a>Testing Environment</h2>
<p>The Asterisk SCF project utilizes a very comprehensive, dynamic, and highly automated testing environment that helps to ensure high quality and high performance levels by utilizing build tests, unit tests, integration tests, and system tests.</p>
<p>Automated build and unit testing is done when commits to the source-code are made. Additionally, an exhaustive set of tests covering component, integration, and system testing is done on a nightly basis using an automated virtualization environment. This nightly-testing produces data from the various tools that is then collected, checked, and then those results along with network capture files, call flows, etc. are all fed back into the Bamboo job where they are then saved as artifacts for reporting purposes, issue triage, and for possible use in subsequent stages within the test run.</p>
<map name='GLIFFY_MAP_12550412_flow_chart_test_env'></map>
<table width="100%">
<tr>
<td align="center">
<table>
<caption align="bottom">
</caption>
<tr>
<td>
<img style="border: none; width: 687px; height: 673px;"
usemap="#GLIFFY_MAP_12550412_flow_chart_test_env"
src="/wiki/download/attachments/12550412/flow+chart+test+env.png?version=1&modificationDate=1299859166642"
alt="A&#32;Gliffy&#32;Diagram&#32;named&#58;&#32;flow&#32;chart&#32;test&#32;env"/>
</td>
</tr>
</table>
</td>
</tr>
</table>
<h2><a name="Testing-3rdPartyTools"></a>3rd Party Tools</h2>
<h4><a name="Testing-AutomatedBuilding%2CTesting%2CandReporting"></a>Automated Building, Testing, and Reporting</h4>
<ul>
<li><a href="http://www.atlassian.com/software/bamboo/" class="external-link" rel="nofollow">Bamboo</a></li>
</ul>
<h4><a name="Testing-Virtualization"></a>Virtualization</h4>
<ul>
<li><a href="http://www.vmware.com/products/vsphere-hypervisor/index.html" class="external-link" rel="nofollow">vSphere</a></li>
</ul>
<h4><a name="Testing-NetworkMonitoring%2FPacketCapture"></a>Network Monitoring / Packet Capture</h4>
<ul>
<li><a href="http://www.wireshark.org/" class="external-link" rel="nofollow">Wireshark</a></li>
<li><a href="http://ngrep.sourceforge.net/" class="external-link" rel="nofollow">ngrep</a></li>
</ul>
<h4><a name="Testing-SIP"></a>SIP</h4>
<ul>
<li><a href="http://sipp.sourceforge.net/" class="external-link" rel="nofollow">SIPp</a></li>
<li><a href="http://sourceforge.net/projects/callflow/" class="external-link" rel="nofollow">Callflow</a></li>
<li><a href="http://freshmeat.net/projects/sipbomber/" class="external-link" rel="nofollow">Sipbomber</a></li>
<li><a href="https://www.ee.oulu.fi/research/ouspg/PROTOS_Test-Suite_c07-sip" class="external-link" rel="nofollow">PROTOS Test Suite</a></li>
<li><a href="http://www.pjsip.org/pjsip/docs/html/page_pjsip_perf_c.htm" class="external-link" rel="nofollow">pjsip-perf</a></li>
<li><a href="http://mitesterforsip.sourceforge.net/" class="external-link" rel="nofollow">miTester for SIP</a></li>
</ul>
<h2><a name="Testing-TestPlans"></a>Test Plans</h2>
<ul><li><a href="/wiki/display/TOP/Test+Environment" title="Test Environment">Test Environment</a></li><li><a href="/wiki/display/TOP/Asterisk+SCF+v1.0+Beta+-+Test+Plan" title="Asterisk SCF v1.0 Beta - Test Plan">Asterisk SCF v1.0 Beta - Test Plan</a></li></ul>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;">
<a href="https://wiki.asterisk.org/wiki/users/viewnotifications.action" class="grey">Change Notification Preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Testing">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=12550412&revisedVersion=8&originalVersion=7">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Testing?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>