<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/3405/">https://reviewboard.asterisk.org/r/3405/</a>
</td>
</tr>
</table>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/3405/diff/3/?file=57409#file57409line49" style="color: black; font-weight: bold; text-decoration: underline;">branches/12/include/asterisk/spinlock.h</a>
<span style="font-weight: normal;">
(Diff revision 3)
</span>
</th>
</tr>
</thead>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">49</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="n">AST_SPINLOCK_TYPE_GAS_X86</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">50</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="n">AST_SPINLOCK_TYPE_GAS_ARM</span><span class="p">,</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">51</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="n">AST_SPINLOCK_TYPE_GAS_SPARC</span><span class="p">,</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Spinlocks are useless on a non-multicore machines. The assembly versions should have some check to see if the machine the code is being built for is multicore.</pre>
</div>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/3405/diff/3/?file=57409#file57409line81" style="color: black; font-weight: bold; text-decoration: underline;">branches/12/include/asterisk/spinlock.h</a>
<span style="font-weight: normal;">
(Diff revision 3)
</span>
</th>
</tr>
</thead>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">81</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="k">while</span> <span class="p">(</span><span class="n">__sync_lock_test_and_set</span><span class="p">(</span><span class="n">lock</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span> <span class="k">while</span><span class="p">(</span><span class="o">*</span><span class="n">lock</span><span class="p">)</span> <span class="p">;</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Always use curly braces.</pre>
</div>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/3405/diff/3/?file=57409#file57409line128" style="color: black; font-weight: bold; text-decoration: underline;">branches/12/include/asterisk/spinlock.h</a>
<span style="font-weight: normal;">
(Diff revision 3)
</span>
</th>
</tr>
</thead>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">128</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="k">while</span> <span class="p">(</span><span class="n">x86chgl</span><span class="p">(</span><span class="n">lock</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span> <span class="k">while</span><span class="p">(</span><span class="o">*</span><span class="n">lock</span><span class="p">);</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Curlies</pre>
</div>
<br />
<div>
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/3405/diff/3/?file=57409#file57409line399" style="color: black; font-weight: bold; text-decoration: underline;">branches/12/include/asterisk/spinlock.h</a>
<span style="font-weight: normal;">
(Diff revision 3)
</span>
</th>
</tr>
</thead>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">399</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="cm">/*!</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">400</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="cm"> * \brief Initialize a spin lock</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">401</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="cm"> * \param lock Address of the lock</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">402</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="cm"> * \retval 0 Success</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">403</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="cm"> * \retval other Failure</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">404</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="cm"> */</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">405</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="k">static</span> <span class="n">force_inline</span> <span class="kt">int</span> <span class="nf">ast_spinlock_init</span><span class="p">(</span><span class="kt">ast_spinlock_t</span> <span class="o">*</span><span class="n">lock</span><span class="p">);</span></pre></td>
</tr>
</tbody>
</table>
<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I suppose a comment saying the below prototypes are to ensure that the spinlock implementations above provice the same API.</pre>
</div>
<br />
<p>- rmudgett</p>
<br />
<p>On April 17th, 2014, 12:26 p.m. CDT, George Joseph wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers, Corey Farrell, Joshua Colp, and rmudgett.</div>
<div>By George Joseph.</div>
<p style="color: grey;"><i>Updated April 17, 2014, 12:26 p.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="https://issues.asterisk.org/jira/browse/ASTERISK-23553">ASTERISK-23553</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
Asterisk
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Now ready for prime time...
This patch adds support for spinlocks in Asterisk.
There are cases in Asterisk where it might be desirable to lock a short critical code section but not incur the context switch and yield penalty of a mutex or rwlock. The primary spinlock implementations execute exclusively in userspace and therefore don't incur those penalties. Spinlocks are NOT meant to be a general replacement for mutexes. They should be used only for protecting short blocks of critical code such as simple compares and assignments. Operations that may block, hold a lock, or cause the thread to give up it's timeslice should NEVER be attempted in a spinlock.
The first use case for spinlocks is in astobj2 - internal_ao2_ref. Currently the manipulation of the reference counter is done with an ast_atomic_fetchadd_int which works fine. When weak reference containers are introduced however, there's an additional comparison and assignment that'll need to be done while the lock is held. A mutex would be way too expensive here, hence the spinlock. Given that lock contention in this situation would be infrequent, the overhead of the spinlock is only a few more machine instructions than the current ast_atomic_fetchadd_int call.
7 implementations were tested on various i366, x86_64, ARM and Sparc platforms running Linux, FreeBSD, OSX and Solaris. The test suite and results can be found here... https://github.com/gtjoseph/spintest
Summary...
Tested various spinlock implementations.
+ GCC Atomics gcc_atomics
+ x86 assembly gas_x86
+ ARM assembly gas_arm
+ Sparc assembly gas_sparc
+ OSX Atomics osx_atomics
+ pthreads spinlock pthread_spinlock
+ pthreads mutex pthread_mutex
+ pthreads adaptive mutex pthread_mutex_adaptive_np
Not all implementations were available on all platforms. For instance, the gas_x86 implementation won't be available on an arm or sparc platform.
Each implementation was tested with the same locking scenario which is a short block of compares and assignments representing the most anticipated Asterisk use cases. The test case was run in a tight loop of 25,000,000 iterations executed in parallel by 1..5 simultaneous threads.
The test suite was run on the following platforms:
+ Darwin-OSX-x86_64-2core
+ FreeBSD-BSD-amd64-4core
+ FreeBSD-BSD-i386-1core
+ Linux-CentOS-i686-1core
+ Linux-Debian-i686-4core
+ Linux-Debian-x86_64-4core
+ Linux-Fedora-armv7l-1core
+ Linux-Fedora-i686-1core
+ Linux-Fedora-x86_64-12core
+ Linux-Fedora-x86_64-1core
+ Linux-Fedora-x86_64-2core
+ Linux-Fedora-x86_64-4core
+ Linux-Fedora-x86_64-8core
+ Linux-Gentoo-sparc64-32core
+ SunOS-Solaris-i86pc-4core
For each platform tested, the real, user and system times were captured in csv form and the final real and system times graphed.
Observations:
+ The GCC Atomics implementation was available on all platforms and generally had the best performance.
+ The native assembly implementations generally performed on par with the GCC Atomics implementation.
+ The pthread_spinlock implementation was not available on Darwin but performed well on the other platforms.
+ The OSX Atomics implementation performed well on Darwin.
+ The pthread_mutex_adaptive implementation was not available on all platforms and it's performance was noticeably worse on the supported platforms.
+ The pthread_mutex implementation was available on all platforms but clearly showed the effects of context switching (high sys time) as lock contention grew.
Conclusions:
+ GCC Atomics should be the first implementation choice.
+ The native assembly implementations should be the second choices.
+ The pthread_spinlock implementation should be the third choice on non-Darwin platforms.
+ OSX Atomics should be the third choice on Darwin.
+ The pthread_mutex_adaptive implementation should be eliminated.
+ The pthread_mutex implementation should be the implementation of last resort.
+ If none of the implementations are available, a compilation #error will be raised.
</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">See above.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>branches/12/include/asterisk/spinlock.h <span style="color: grey">(PRE-CREATION)</span></li>
<li>branches/12/include/asterisk/autoconfig.h.in <span style="color: grey">(412427)</span></li>
<li>branches/12/configure.ac <span style="color: grey">(412427)</span></li>
<li>branches/12/configure <span style="color: grey">(UNKNOWN)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/3405/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>