<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/1408/">https://reviewboard.asterisk.org/r/1408/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On September 2nd, 2011, 8:47 p.m., <b>Tilghman Lesher</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href="https://reviewboard.asterisk.org/r/1408/diff/2/?file=19934#file19934line371" style="color: black; font-weight: bold; text-decoration: underline;">/res/res_config_sqlite3.c</a>
<span style="font-weight: normal;">
(Diff revision 2)
</span>
</th>
</tr>
</thead>
<tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
<tr>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<td colspan="2"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">static int row_to_varlist(void *arg, int num_columns, char **values, char **columns)</pre></td>
</tr>
</tbody>
<tbody>
<tr>
<th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" align="right"><font size="2"></font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>
<th bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" align="right"><font size="2">371</font></th>
<td bgcolor="#c5ffc4" width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "> <span class="cm">/* reverse order, but who cares? */</span></pre></td>
</tr>
</tbody>
</table>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">The people who deliberately order disallow/allow columns or permit/deny columns care.</pre>
</blockquote>
<p>On September 3rd, 2011, 12:26 a.m., <b>Terry Wilson</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is callback function for a single row of results and it is just the reverse order of the columns in that row. We aren't guaranteed what order the columns in the database are presented, so altering their order in the variable list shouldn't matter, especially since it is a list of column names and values. Someone seeking a particular value will parse the list comparing on the column name.</pre>
</blockquote>
<p>On September 3rd, 2011, 9:08 a.m., <b>Tilghman Lesher</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Take a look at the SIP and IAX drivers, which parse the values that come back from realtime in the order in which they are presented. I agree that it's a bad precedent that we have had to order columns in the table to match how they're dealt with in the driver; however, it is precedent nonetheless. If you're going to break this assumption (that is true in every other realtime driver), then that needs to be clearly documented (on the Wiki, since there's really no inline documentation for realtime drivers) for this driver.</pre>
</blockquote>
<p>On September 3rd, 2011, 10:51 a.m., <b>Terry Wilson</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I looked at chan_sip/iax2 and I really don't see anywhere where the results from ast_load_realtime() are not parsed by doing a strcasecmp on the 'name' field. Since the select queries on every backend use "SELECT *" there is no way to tell what order these columns are going to come back. It often depends on what order they were actually created at CREATE TABLE time. If someone ever relied on that order, they deserve to fail. But it would be really hard to do so. Who is going to traverse a linked list, keeping a counter, and do something like "if (i != 3) continue;". That just isn't how people use linked lists. If this was a multi-row query (which requires an ast_config) I would completely agree that order matters. On this, though, I just can't imagine a way where someone would be silly enough to go to the extra work it would require to make the code brittle. It isn't that hard to change, but inserting at the head is more efficient and as far as the code is concerned, column order in a SELECT * is undefined. We can not know what order the values were listed in CREATE TABLE. Warning someone about it in the wiki would be like warning someone that the random number produced by X is going to be different than the one produced by Y.</pre>
</blockquote>
<p>On September 3rd, 2011, 4:05 p.m., <b>Tilghman Lesher</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I think you're missing the point. I'm not saying that drivers depend upon fields being positionally arranged. I'm saying that people intentionally structure their tables with the field "disallow" previous to "allow", so they can do disallow=all; allow=ulaw,gsm. If in your driver, suddenly the fields are returned in reverse order, the user will have all codecs disabled. We don't currently allow prefixing codecs with a '!' to do a disallow (allow=all,!g729) in the allow column (and perhaps we should, to deal with this specific problem), but it's still precedent that means we have to honor the user's definition and present fields in the order in which they were defined.
Otherwise, what you're saying here is... all realtime drivers return columns in the order in which they are defined... except this one peculiar driver, which returns them in the reverse order. If nothing else, it violates the principle of least surprise.</pre>
</blockquote>
</blockquote>
<pre style="margin-left: 1em; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Ok, I see what you are saying. That is awful, but I can see how the current situation with allow and disallow parsing depending upon order in the config file and (non-static) realtime not having any way to determine that order since it would be a single column with semi-colon separated values in allow and disallow and no defined order since we have to do a select *. That is ugly. If someone decided they wanted to go the other way (allow=all, disallow=g729) they would have to recreate their freaking table and re-import all of the data (or create a view on the existing table, maybe). Anyway, wow. I'll go ahead and insert at the tail. Thanks for the explanation.</pre>
<br />
<p>- Terry</p>
<br />
<p>On September 2nd, 2011, 5:05 p.m., Terry Wilson wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers, Olle E Johansson and Tilghman Lesher.</div>
<div>By Terry Wilson.</div>
<p style="color: grey;"><i>Updated Sept. 2, 2011, 5:05 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This patch adds an SQLite 3 realtime driver. It supports multiple databases, ast_realtime_require (warn, createclose, and createchar support including the ability to create missing tables), static realtime, and batching commits in transactions to increase write performance. It currently caches no table data, but I figure that premature optimization is bad anyway.
It, like other realtime drivers, doesn't escape data when building the SQL strings. I started out using parameter binding, but SQLite doesn't let you bind column names, so given how dynamic everything with realtime is, I just when with building the statements outright. I could manually escape the parameters, but the only thing in the SQLite 3 library I could find that does that dynamically allocates the string and that ends up being a lot of little allocs and frees. I suppose we could write our own using a single ast_str with some ast_str_reset calls between each one, but it is a bit of a pain. Until we want to open up the realtime api to things like AMI where user-generated input is going to be a problem, escaping isn't really necessary.
The impetus for writing this is that Asterisk now has a built-in SQLite 3 database which the astdb uses. The next step would be to convert the astdb to using the realtime calls, thus making it possible for it to be used with any realtime backend (but using the default SQLite 3 db unless configured otherwise). After that, we can start rewriting some things that use the astdb in very crude ways to start using realtime calls to the internal (or external, if configured that way) databases so that the data can be stored in more logical ways than a key/value store allows.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Verified that static realtime extensions.conf loads properly. Realtime CLI commands all behave as expected. Tested unload/load and reload scenarios making sure that they behaved as expected and that there were no ref/memory leaks. Made sure that batching worked.
Once we move the astdb to using realtime calls, the existing unit tests for that should give a workout to the realtime driver as well. I didn't write a new test pointing to the astdb because the astdb locks its database so it cannot be written to from another connection.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/trunk/configs/res_config_sqlite3.conf.sample <span style="color: grey">(PRE-CREATION)</span></li>
<li>/trunk/res/res_config_sqlite3.c <span style="color: grey">(PRE-CREATION)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/1408/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>