<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/2042/1/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Developer+Driven+Automated+Testing">Developer Driven Automated Testing</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~dlee">David M. Lee</a>
</h4>
<div id="versionComment">
<b>Comment:</b>
Updated CTest function names to Kevin's recent additions of awesomeness<br />
</div>
<br/>
<h4>Changes (2)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" >Unit tests are built as standalone applications. The Boost Test Library will generate a {{main}} (in the file where {{BOOST_TEST_MODULE}} is defined) which can handle command line parsing, running tests, displaying results, exiting with a non-zero return code on failure, etc. <br> <br></td></tr>
<tr><td class="diff-changed-lines" >Add the test to CTest using <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">{{boost_add_test(_\[TestNameHere]_)}},</span> <span class="diff-added-words"style="background-color: #dfd;">{{asterisk_scf_test_boost(_\[TestNameHere]_)}},</span> so that the test gets run during {{make test}}. You should also be able to run your test directly for debugging purposes. <br></td></tr>
<tr><td class="diff-unchanged" > <br>h2. Component Tests <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" >You must also write an Ice config file that specifies the services and their configuration needed for running the test. Note that service startup order is by default unspecified, so you need to add {{IceBox.LoadOrder}} to specify component load order. <br> <br></td></tr>
<tr><td class="diff-changed-lines" >Add the test to CTest using <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">{{icebox_add_test(_\[TestNameHere]_</span> <span class="diff-added-words"style="background-color: #dfd;">{{asterisk_scf_test_icebox(_\[TestNameHere]_</span> _\[test-config-file-here.conf]_)}}. In order to run the test directly for debugging purposes, you have to actually launch icebox with the {{--Ice.Config}} option to specify the test config file. <br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<p>As described on <a href="/wiki/display/TOP/Asterisk+SCF+Testing" title="Asterisk SCF Testing">Asterisk SCF Testing</a>, we break down developer driven <a href="http://en.wikipedia.org/wiki/Test_automation" class="external-link" rel="nofollow">automated tests</a> into several categories, including <a href="http://en.wikipedia.org/wiki/Unit_testing" class="external-link" rel="nofollow">unit tests</a>. The purpose of this page is to describe how a developer goes about implementing the different varieties of automated tests.</p>
<div>
<ul>
<li><a href='#DeveloperDrivenAutomatedTesting-Fundamentals'>Fundamentals</a></li>
<ul>
<li><a href='#DeveloperDrivenAutomatedTesting-Boost%3A%3ATest'>Boost::Test</a></li>
<li><a href='#DeveloperDrivenAutomatedTesting-CTest'>CTest</a></li>
</ul>
<li><a href='#DeveloperDrivenAutomatedTesting-WritingDeveloperTests'>Writing Developer Tests</a></li>
<ul>
<li><a href='#DeveloperDrivenAutomatedTesting-UnitTests'>Unit Tests</a></li>
<li><a href='#DeveloperDrivenAutomatedTesting-ComponentTests'>Component Tests</a></li>
<li><a href='#DeveloperDrivenAutomatedTesting-IntegrationTests'>Integration Tests</a></li>
</ul>
</ul></div>
<h1><a name="DeveloperDrivenAutomatedTesting-Fundamentals"></a>Fundamentals</h1>
<h2><a name="DeveloperDrivenAutomatedTesting-Boost%3A%3ATest"></a>Boost::Test</h2>
<p>The <a href="http://www.boost.org/doc/libs/1_34_0/libs/test/doc/index.html" class="external-link" rel="nofollow">Boost Test Library</a> is the unit testing framework we use for driving our unit tests. While we encourage you to read the documentation, here are some notes about our usage of the Boost Test Library.</p>
<ul>
<li><tt>BOOST_AUTO_TEST_SUITE</tt> and <tt>BOOST_AUTO_TEST_CASE</tt> provide a concise DSL for writing your tests. Take advantage of that.</li>
<li>We define <tt>BOOST_ALL_NO_LIB</tt> and <tt>BOOST_ALL_DYN_LINK</tt> in our build scripts. This means the preferred method for bringing in the Boost Test Library is to:
<ol>
<li><tt>#include <boost/test/unit_test.hpp></tt></li>
<li>In exactly one file in your test app/component, prior to the <tt>unit_test.hpp</tt> include, <tt>#define BOOST_TEST_MODULE <em>[TestNameHere]</em></tt></li>
<li>In your CMake script, add the Boost <tt>unit_test_framework</tt> library.</li>
</ol>
</li>
</ul>
<h2><a name="DeveloperDrivenAutomatedTesting-CTest"></a>CTest</h2>
<p>While we use the Boost Test Library for writing our tests, CMake has a built-in execution framework for executing the tests as a part of the build: <a href="http://www.cmake.org/cmake/help/ctest-2-8-docs.html" class="external-link" rel="nofollow">CTest</a>. There's actually not a whole lot to CTest; it just provides a way for launching applications, handling the output, and failing the build if the application stops with a non-zero exit code.</p>
<ul>
<li>Tests are invoked using <tt>make test</tt>.</li>
<li>These automated tests should be self contained, meaning that they require no manual setup prior to execution. Except, of course, for <tt>make all</tt> <img class="emoticon" src="/wiki/images/icons/emoticons/smile.gif" height="20" width="20" align="absmiddle" alt="" border="0"/>
<ul>
<li>I repeat: the user should not have to setup components, files, databases, enchantments, charms or perform any other rituals in order to successfully run the developer written tests. They must be able to:
<ol>
<li>Check out from source control</li>
<li>Run CMake to configure the build</li>
<li><tt>make all test</tt></li>
<li>See which tests passed, and which tests failed</li>
</ol>
</li>
</ul>
</li>
<li>CTest considers each execution to be an individual test. The reality is that each execution actually runs a series of tests.
<ul>
<li>CTest will only tell you which execution failed. Run <tt>make test ARGS=-V</tt> to see Boost Test output, in order to find out exactly which test failed.</li>
</ul>
</li>
</ul>
<h1><a name="DeveloperDrivenAutomatedTesting-WritingDeveloperTests"></a>Writing Developer Tests</h1>
<h2><a name="DeveloperDrivenAutomatedTesting-UnitTests"></a>Unit Tests</h2>
<p><a href="/wiki/display/TOP/Asterisk+SCF+Testing#AsteriskSCFTesting-UnitTesting">Unit tests</a> are the simplest to write, since they test individual classes. Usually there's no need to startup Ice, other components, or lots of infrastructure. Classes that are the easiest to unit test are ones that have few dependencies (like small utility classes) or only depend on abstract base classes (which allow you to inject a <a href="http://xunitpatterns.com/Test%20Double.html" class="external-link" rel="nofollow">Test Double</a> for specific purpose of running the test).</p>
<p>Unit tests are built as standalone applications. The Boost Test Library will generate a <tt>main</tt> (in the file where <tt>BOOST_TEST_MODULE</tt> is defined) which can handle command line parsing, running tests, displaying results, exiting with a non-zero return code on failure, etc.</p>
<p>Add the test to CTest using <tt>asterisk_scf_test_boost(<em>[TestNameHere]</em>)</tt>, so that the test gets run during <tt>make test</tt>. You should also be able to run your test directly for debugging purposes.</p>
<h2><a name="DeveloperDrivenAutomatedTesting-ComponentTests"></a>Component Tests</h2>
<div class='panelMacro'><table class='infoMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/wiki/images/icons/emoticons/information.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td>todo. probably similar to integration tests.</td></tr></table></div>
<h2><a name="DeveloperDrivenAutomatedTesting-IntegrationTests"></a>Integration Tests</h2>
<p>Integration tests are a bit more complicated, because they require wiring up several components in order to run the test. They test complete components, instead of individual classes. The tests interact with the components under test strictly via the interfaces defined in the .ice files. However, there may be special interfaces that are only provided for test builds to allow for <a href="http://en.wikipedia.org/wiki/White-box_testing" class="external-link" rel="nofollow">white box testing</a>.</p>
<p>Since components require <a href="http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/IceBox.html#22064" class="external-link" rel="nofollow">Icebox</a> to run, the integration tests themselves must be written as an Icebox component. This takes some special consideration for both the Boost Test Library and CTest.</p>
<ul>
<li>In the file where <tt>BOOST_TEST_MODULE</tt> is defined, also define <tt>BOOST_TEST_NO_MAIN</tt> to inhibit the generation of the typical <tt>main</tt> function.</li>
<li>Your <tt>IceBox::Service</tt>'s <tt>start</tt> member function must call <tt>boost::unit_test::unit_test_main</tt> to run the test suite. It must then call <tt>exit</tt> with the return code in order to promptly shutdown with the proper return code.</li>
</ul>
<p>You must also write an Ice config file that specifies the services and their configuration needed for running the test. Note that service startup order is by default unspecified, so you need to add <tt>IceBox.LoadOrder</tt> to specify component load order.</p>
<p>Add the test to CTest using <tt>asterisk_scf_test_icebox(<em>[TestNameHere]</em> <em>[test-config-file-here.conf]</em>)</tt>. In order to run the test directly for debugging purposes, you have to actually launch icebox with the <tt>--Ice.Config</tt> option to specify the test config file.</p>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;">
<a href="https://wiki.asterisk.org/wiki/users/viewnotifications.action" class="grey">Change Notification Preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Developer+Driven+Automated+Testing">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=12550835&revisedVersion=2&originalVersion=1">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Developer+Driven+Automated+Testing?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>