<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/en/2171/18/9/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Testing+and+Asterisk+SCF">Testing and Asterisk SCF</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~beagles">Brent Eagles</a>
</h4>
<br/>
<h4>Changes (31)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-changed-lines" >{warning:title=In Progress} <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">{warning}</span> <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">{warning} <br></td></tr>
<tr><td class="diff-unchanged" > <br>h3. Slide Suggestions <br> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color:white} <br></td></tr>
<tr><td class="diff-unchanged" >{panel:title=Testing and Asterisk SCF|titleBGColor=#2022FF|bgColor=#7788FF} <br></td></tr>
<tr><td class="diff-changed-lines" >* <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Testing</span> is an integral part of development and the <span class="diff-changed-words">process.<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-changed-lines" >* <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Builds</span> and running of tests are <span class="diff-changed-words">automated.<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">* boost::test test framework. <br>* Different levels of testing. <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">* {color:white}boost::test test framework.{color} <br>* {color:white}Different levels of testing.{color} <br></td></tr>
<tr><td class="diff-changed-lines" >** <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Unit<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-changed-lines" >** <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Component<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-changed-lines" >** <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>System<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-unchanged" >{panel} <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color} <br></td></tr>
<tr><td class="diff-unchanged" > <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color:white} <br></td></tr>
<tr><td class="diff-unchanged" >{panel:title=Testing and Asterisk SCF - Part of the Process|titleBGColor=#2022FF|bgColor=#7788FF} <br></td></tr>
<tr><td class="diff-changed-lines" >* <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Each</span> component is expected to have a test <span class="diff-changed-words">suite.<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-changed-lines" >* <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Tests</span> are run automatically as part of the automated build/test <span class="diff-changed-words">process.<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">* Test-ability *is* a design consideration. <br>* Tests are included in code reviews. <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">* {color:white}Test-ability{color} {color:white}{*}is{*}{color} {color:white}a design consideration.{color} <br>* {color:white}Tests are included in code reviews.{color} <br></td></tr>
<tr><td class="diff-unchanged" >{panel} <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color} <br></td></tr>
<tr><td class="diff-unchanged" > <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color:white} <br></td></tr>
<tr><td class="diff-unchanged" >{panel:title=Testing and Asterisk SCF - Test Types|titleBGColor=#2022FF|bgColor=#7788FF} <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">* {color:white}Unit tests are typically in standalone executables and focus on small sections of high risk or fundamental code.{color} <br>* {color:white}Component tests load the component using a real-world configuration along with other supporting component and usually contain:{color} <br>** {color:white}functional tests{color} <br>** {color:white}unit tests where mock objects would be required but are impractical{color} <br>** {color:white}not-quite-functional-or-unit tests that are small and focused but involve strings of components in contrived, artificial situations{color} <br>* {color:white}Some components are impractical to test with unit and even component tests (e.g. SIP) This component is most vigorously tested at the system level which brings us to..{color} <br></td></tr>
<tr><td class="diff-unchanged" > <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">* Unit tests are typically in standalone executables and focus on small sections of high risk or fundamental code. <br>* Component tests load the component using a real-world configuration along with other supporting component and usually contain: <br>** functional tests <br>** unit tests where mock objects would be required but are impractical <br>** not-quite-functional-or-unit tests that are small and focused but involve strings of components in contrived, artificial situations <br>* Some components are impractical to test with unit and even component tests (e.g. SIP) This component is most vigorously tested at the system level which brings us to.. <br> <br></td></tr>
<tr><td class="diff-unchanged" >{panel} <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color} <br></td></tr>
<tr><td class="diff-unchanged" > <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{color:white} <br></td></tr>
<tr><td class="diff-unchanged" >{panel:title=Testing and Asterisk SCF - Higher Level Testing|titleBGColor=#2022FF|bgColor=#7788FF} <br></td></tr>
<tr><td class="diff-changed-lines" >* <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Useful</span> automated testing of a distributed, interactive, concurrent system <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">\</span>!= trivial.<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-changed-lines" >* <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Enter</span> the Asterisk SCF System Testing <span class="diff-changed-words">Framework<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">** plugin architecture with plugins for: <br>| configuration | component life-cycle control | <br>| fail-over simulation | external device simulation (SIP) | <br>| remote control | system and post-mortem analysis | <br></td></tr>
<tr><td class="diff-changed-lines" >** <span class="diff-changed-words"><span class="diff-added-chars"style="background-color: #dfd;">{color:white}</span>Heavy</span> lifting for performance and stress testing as well as full system functional <span class="diff-changed-words">testing.<span class="diff-added-chars"style="background-color: #dfd;">{color}</span></span> <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">** {color:white}Facilitating analysis is a primary design goal.{color} <br>** {color:white}Automatically generates detailed test specifications including commands and configuration. {color} <br>** {color:white}Implemented in Python and coordinates tests across multiple platforms. {color} <br>** {color:white}Can be run standalone in your own development environment. {color} <br>{panel} <br></td></tr>
<tr><td class="diff-unchanged" > <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">* https://wiki.asterisk.org/wiki/display/TOP/Testing for more info <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">{color:white} <br>{panel:title=Testing and Asterisk SCF - More About Plugins|titleBGColor=#2022FF|bgColor=#7788FF} <br>* {color:white}plugin architecture with plugins for:{color} <br>| {color:#ffffff}configuration{color} | {color:#ffffff}component life-cycle control{color} | <br>| {color:#ffffff}fail-over simulation{color} | {color:#ffffff}external device simulation (SIP){color} | <br>| {color:#ffffff}remote control{color} | {color:#ffffff}system and post-mortem analysis{color} | <br>* Plugin spotlight <br>** PROTOS - independently developed INVITE focused torture test <br>** SIPp - automated, configurable simulation of call scenarios <br>** Wireshark - on the wire logging of SIP, RTP, T38, etc. and generation of call graphs <br></td></tr>
<tr><td class="diff-unchanged" >{panel} <br>{color} <br> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">_Maybe some snapshots of sip call graphs, log output, etc_. <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">{color:white} <br>{panel:title=Testing and Asterisk SCF - Output Examples|titleBGColor=#2022FF|bgColor=#7788FF} <br>* _insert SIP call graph_ <br>* _screen shot of build/run artifact or something similar.. has to be cool!_ <br>{panel} <br>{color} <br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<div class='panelMacro'><table class='warningMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/wiki/images/icons/emoticons/forbidden.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>In Progress</b><br /></td></tr></table></div>
<h3><a name="TestingandAsteriskSCF-SlideSuggestions"></a>Slide Suggestions</h3>
<div class="panel" style="background-color: #7788FF;border-width: 1px;"><div class="panelHeader" style="border-bottom-width: 1px;background-color: #2022FF;"><b>Testing and Asterisk SCF</b></div><div class="panelContent" style="background-color: #7788FF;">
<ul>
<li><font color="white">Testing is an integral part of development and the process.</font></li>
<li><font color="white">Builds and running of tests are automated.</font></li>
<li><font color="white">boost::test test framework.</font></li>
<li><font color="white">Different levels of testing.</font>
<ul>
<li><font color="white">Unit</font></li>
<li><font color="white">Component</font></li>
<li><font color="white">System</font></li>
</ul>
</li>
</ul>
</div></div>
<div class="panel" style="background-color: #7788FF;border-width: 1px;"><div class="panelHeader" style="border-bottom-width: 1px;background-color: #2022FF;"><b>Testing and Asterisk SCF - Part of the Process</b></div><div class="panelContent" style="background-color: #7788FF;">
<ul>
<li><font color="white">Each component is expected to have a test suite.</font></li>
<li><font color="white">Tests are run automatically as part of the automated build/test process.</font></li>
<li><font color="white">Test-ability</font> <font color="white"><b>is</b></font> <font color="white">a design consideration.</font></li>
<li><font color="white">Tests are included in code reviews.</font></li>
</ul>
</div></div>
<div class="panel" style="background-color: #7788FF;border-width: 1px;"><div class="panelHeader" style="border-bottom-width: 1px;background-color: #2022FF;"><b>Testing and Asterisk SCF - Test Types</b></div><div class="panelContent" style="background-color: #7788FF;">
<ul>
<li><font color="white">Unit tests are typically in standalone executables and focus on small sections of high risk or fundamental code.</font></li>
<li><font color="white">Component tests load the component using a real-world configuration along with other supporting component and usually contain:</font>
<ul>
<li><font color="white">functional tests</font></li>
<li><font color="white">unit tests where mock objects would be required but are impractical</font></li>
<li><font color="white">not-quite-functional-or-unit tests that are small and focused but involve strings of components in contrived, artificial situations</font></li>
</ul>
</li>
<li><font color="white">Some components are impractical to test with unit and even component tests (e.g. SIP) This component is most vigorously tested at the system level which brings us to..</font></li>
</ul>
</div></div>
<div class="panel" style="background-color: #7788FF;border-width: 1px;"><div class="panelHeader" style="border-bottom-width: 1px;background-color: #2022FF;"><b>Testing and Asterisk SCF - Higher Level Testing</b></div><div class="panelContent" style="background-color: #7788FF;">
<ul>
<li><font color="white">Useful automated testing of a distributed, interactive, concurrent system != trivial.</font></li>
<li><font color="white">Enter the Asterisk SCF System Testing Framework</font>
<ul>
<li><font color="white">Heavy lifting for performance and stress testing as well as full system functional testing.</font></li>
<li><font color="white">Facilitating analysis is a primary design goal.</font></li>
<li><font color="white">Automatically generates detailed test specifications including commands and configuration. </font></li>
<li><font color="white">Implemented in Python and coordinates tests across multiple platforms. </font></li>
<li><font color="white">Can be run standalone in your own development environment. </font></li>
</ul>
</li>
</ul>
</div></div>
<p><font color="white"><br/>
<div class="panel" style="background-color: #7788FF;border-width: 1px;"><div class="panelHeader" style="border-bottom-width: 1px;background-color: #2022FF;"><b>Testing and Asterisk SCF - More About Plugins</b></div></div><br/>
* <font color="white"></font>plugin architecture with plugins for:</font></p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<td class='confluenceTd'> <font color="#ffffff">configuration</font> </td>
<td class='confluenceTd'> <font color="#ffffff">component life-cycle control</font> </td>
</tr>
<tr>
<td class='confluenceTd'> <font color="#ffffff">fail-over simulation</font> </td>
<td class='confluenceTd'> <font color="#ffffff">external device simulation (SIP)</font> </td>
</tr>
<tr>
<td class='confluenceTd'> <font color="#ffffff">remote control</font> </td>
<td class='confluenceTd'> <font color="#ffffff">system and post-mortem analysis</font> </td>
</tr>
</tbody></table>
</div>
<ul>
<li>Plugin spotlight
<ul>
<li>PROTOS - independently developed INVITE focused torture test</li>
<li>SIPp - automated, configurable simulation of call scenarios</li>
<li>Wireshark - on the wire logging of SIP, RTP, T38, etc. and generation of call graphs
<div class="panel" style="border-width: 1px;"><div class="panelContent">
<p><font color=""></font></p>
<p><font color="white"></font></p>
<div class="panel" style="background-color: #7788FF;border-width: 1px;"><div class="panelHeader" style="border-bottom-width: 1px;background-color: #2022FF;"><b>Testing and Asterisk SCF - Output Examples</b></div></div>
<ul>
<li><em>insert SIP call graph</em></li>
<li><em>screen shot of build/run artifact or something similar.. has to be cool!</em></li>
</ul>
</div></div>
<p><font color=""></font></p></li>
</ul>
</li>
</ul>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;" class="grey">
<a href="https://wiki.asterisk.org/wiki/users/removespacenotification.action?spaceKey=TOP">Stop watching space</a>
<span style="padding: 0px 5px;">|</span>
<a href="https://wiki.asterisk.org/wiki/users/editmyemailsettings.action">Change email notification preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Testing+and+Asterisk+SCF">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=19006242&revisedVersion=3&originalVersion=2">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Testing+and+Asterisk+SCF?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>