<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/4108/">https://reviewboard.asterisk.org/r/4108/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On March 31st, 2015, 5:52 p.m. EDT, <b>rmudgett</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This approach is much simpler and easier to understand than previous attempts. Since it is more general it could be used to avoid mutual ref issues such as between the stasis_topic and stasis_subscription as described in main/stasis.c.
All of the issues I've pointed out are minor.</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">With that said I've updated the API to include REF_DEBUG parameters.
One thing with REF_DEBUG, I don't think there is a reasonable way to provide procedures that don't accept tag/file/line/func parameters. The real overhead of REF_DEBUG comes from fprintf / fflush, so I don't think this matters. Especially given that internal_ao2_ref already takes 3 of the 4 parameters.
I think it's worth looking at getting rid of the dual ABI of ao2 (will need to benchmark). It would be useful for example to have REF_DEBUG information known by INTERNAL_OBJ so we get more useful error messages. It would also simplify code by removing a lot of the #if/#else blocks throughout Asterisk.</pre>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On March 31st, 2015, 5:52 p.m. EDT, <b>rmudgett</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/4108/diff/5/?file=71779#file71779line692" style="color: black; font-weight: bold; text-decoration: underline;">/trunk/include/asterisk/astobj2.h</a>
<span style="font-weight: normal;">
(Diff revision 5)
</span>
</th>
</tr>
</thead>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">692</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="kt">int</span> <span class="nf">__ao2_weakproxy_unsubscribe</span><span class="p">(</span><span class="kt">void</span> <span class="o">*</span><span class="n">weakproxy</span><span class="p">,</span> <span class="n">ao2_weakproxy_notification_cb</span> <span class="n">cb</span><span class="p">,</span> <span class="kt">void</span> <span class="o">*</span><span class="n">data</span><span class="p">,</span> <span class="kt">int</span> <span class="n">locked</span><span class="p">);</span></pre></td>
</tr>
</tbody>
</table>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Why not use the OBJ_NOLOCK flag?</pre>
</blockquote>
</blockquote>
<pre style="margin-left: 1em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I've also added support for OBJ_MULTIPLE in the flag to remove all matching subscriptions.</pre>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On March 31st, 2015, 5:52 p.m. EDT, <b>rmudgett</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/4108/diff/5/?file=71780#file71780line450" style="color: black; font-weight: bold; text-decoration: underline;">/trunk/main/astobj2.c</a>
<span style="font-weight: normal;">
(Diff revision 5)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">static int internal_ao2_ref(void *user_data, int delta, const char *file, int line, const char *func)</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">450</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="tb"> </span><span class="tb"> </span><span class="n">obj</span><span class="o">-></span><span class="n">priv_data</span><span class="p">.</span><span class="n">ref_counter</span><span class="o">++</span><span class="p">;</span></pre></td>
</tr>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">451</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="tb"> </span><span class="tb"> </span><span class="n">internal_weakproxy</span><span class="o">-></span><span class="n">priv_data</span><span class="p">.</span><span class="n">weakptr</span> <span class="o">=</span> <span class="n">obj</span><span class="o">-></span><span class="n">priv_data</span><span class="p">.</span><span class="n">weakptr</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span></pre></td>
</tr>
</tbody>
</table>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Isn't the shell game you are playing with the real object's ref count going to mess up the DEBUG_REF output?
How about this code:
Code assumes that weakproxy ref wasn't bumped when set above.
if weakproxy
if current_value == 1
unlink real and weak
unlock weakproxy
if current_value == 1
run weakproxy callbacks (locking between each callback or locking to run all)
unref weakproxy
unref real</pre>
</blockquote>
<p>On March 31st, 2015, 8:32 p.m. EDT, <b>Corey Farrell</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Lets use realobj to represent the variable in the callers scope (non-ao2 code), user_data is the variable in internal_ao2_ref. This shell game results in REF_DEBUG logs showing the 2nd to last reference being released by ao2_ref(user_data, -1) a couple lines below your finding, just prior to the user releasing the last reference. REF_DEBUG is produced after internal_ao2_ref completes, so it's impossible for the unref's in this block to be logged after ao2_ref(realobj, -1). So without this REF_DEBUG would say that the object was destroyed due to ao2_ref(user_data, -1) before the 2nd to last reference was released.
This REF_DEBUG output was verified using the provided test.</pre>
</blockquote>
</blockquote>
<pre style="margin-left: 1em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">REF_DEBUG output for all objects in test_astobj2_weaken.c is posted to JIRA ticket. I believe this is the desired outcome.</pre>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On March 31st, 2015, 5:52 p.m. EDT, <b>rmudgett</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/4108/diff/5/?file=71780#file71780line798" style="color: black; font-weight: bold; text-decoration: underline;">/trunk/main/astobj2.c</a>
<span style="font-weight: normal;">
(Diff revision 5)
</span>
</th>
</tr>
</thead>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">798</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span class="tb"> </span><span class="n">weakproxy</span> <span class="o">=</span> <span class="n">ao2_alloc</span><span class="p">(</span><span class="n">data_size</span><span class="p">,</span> <span class="n">destructor_fn</span><span class="p">);</span></pre></td>
</tr>
</tbody>
</table>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I think you need to have a weakproxy destructor callback that performs a sanity check on the proxy object to ensure that it is not still linked with a real object.
Or even better, make the ao2_ref code do this check when it is about to destroy a weakproxy object. This check can also destroy anything in the notify list for the case if the real object was not set but subscribers were.</pre>
</blockquote>
<p>On March 31st, 2015, 8:32 p.m. EDT, <b>Corey Farrell</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Once linked to a real object the weakproxy cannot be destroyed until after no references to the real object exist. Remember that until the real object is destroyed it holds a reference to the weakproxy.
As for subscriptions I will have to look into this more, but my intention for subscriptions is a way to request notification as soon as the weakproxy points to NULL. So in the case of adding a subscription when no real object is linked, I think we should run the callback immediately and not add it to the subscription list.</pre>
</blockquote>
</blockquote>
<pre style="margin-left: 1em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I've moved forward with my plan to make callbacks run immediately when weakproxy points to NULL. One use I envision for subscriptions is for ao2_containers. The weakproxy will be linked to a container, then a subscription will be created to unlink weakproxy when it no longer points to a real object. If it already doesn't point to a real object then it should be immediately removed.</pre>
<br />
<p>- Corey</p>
<br />
<p>On March 4th, 2015, 4:43 p.m. EST, Corey Farrell wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers, George Joseph and rmudgett.</div>
<div>By Corey Farrell.</div>
<p style="color: grey;"><i>Updated March 4, 2015, 4:43 p.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
Asterisk
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This implements "weak" references. The weakproxy object is a real ao2 with normal reference counting of its own. When a weakproxy is pointed to a normal object they hold references to each other. The normal object is automatically freed when a single reference remains (the weakproxy). The weakproxy also supports subscriptions that will notify callbacks when the normal object is about to be destroyed.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Ran the included test with REF_DEBUG enabled under valgrind. No reference leaks or improper memory access. Though this does not test for races, I don't know of an automated way to do that.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/trunk/tests/test_astobj2_weaken.c <span style="color: grey">(PRE-CREATION)</span></li>
<li>/trunk/main/astobj2.c <span style="color: grey">(432445)</span></li>
<li>/trunk/include/asterisk/astobj2.h <span style="color: grey">(432445)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/4108/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>