<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/4490/">https://reviewboard.asterisk.org/r/4490/</a>
</td>
</tr>
</table>
<br />
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">The only things I can think of that are issues here are based on theoretical usage of shared DB families, especially global ones.
Say that someone is sharing /foo/key between Asterisk 1 and Asterisk 2. A user on Asterisk 1 sets /foo/key = bar at the same time that a user on Asterisk 2 sets /foo/key = baz. What may happen is that the local Asterisk boxes will set the values locally and then publish the updated values near-simultaneously to the other Asterisk box. The result may be that they both have /foo/key = bar, they both might have /foo/key = baz, or Asterisk 1 may end up with /foo/key = baz and Asterisk 2 ends up with /foo/key = bar. Essentially, there is nothing in place to guarantee the synchronization of globally shared values.
So what's the best way to deal with this? I can think of several answers:
* The AstDB may not be the right tool for this use case and you should be using a key-value store that is built for this instead.
* You may be able to satisfy your use case with unique shared keys instead of global shared keys.
* We may want to make global shared keys one-way. In other words, there is an "owner" of the key and "listeners" for changes in the key's value. Listeners on a global shared key cannot update their local DB with new values; only the owner can push new values to it. Listeners on a global shared key cannot share the value with other listeners; only owners may push changes to listeners.
While I'm thinking about it, there's also the possibility that the underlying AstDB implementation could be replaced with a key-value store that has the concept of replication built into it. If that were done, then presumably, there would be no need to involve Stasis or SIP PUBLISH in the process of sharing values between Asterisk boxes.</pre>
<br />
<p>- Mark Michelson</p>
<br />
<p>On April 6th, 2015, 2:22 a.m. UTC, Matt Jordan wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers.</div>
<div>By Matt Jordan.</div>
<p style="color: grey;"><i>Updated April 6, 2015, 2:22 a.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="https://issues.asterisk.org/jira/browse/ASTERISK-24909">ASTERISK-24909</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
Asterisk
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">As described on the asterisk-dev mailing list [1], I got some inspiration from seeing Kamailio's htable implementation, and thought a similar mechanism would work for the Asterisk Database. This patch is the result.
This patch provides a mechanism to mark families within the AstDB as "shared." The marking of a family as shared is independent of the existance of the family, and is independent of the updates already made to the family. Shared families are subject to distribution with other Asterisk instances, as well as subject to updates from other Asterisk instances. Two strategies for sharing are implemented:
* Global: A 'global' shared family shares the family/key space with all other Asterisk instances. Say we have shared family 'foo', and we have two Asterisk instances. Say the first Asterisk instance (ast1) updates a key in family 'foo' to the following:
ast1
/foo/key = bar
The second Asterisk instance (ast2) would then receive an update in its AstDB:
ast2
/foo/key = bar
If ast2 later updates the same key in its local AstDB to 'foobar', ast1 will receive a notification to update the same key in its AstDB:
ast2
/foo/key = foobar
ast1
/foo/key = foobar
* Unique: A 'unique' shared family shares its values with other Asterisk instances, however, updates from other Asterisk instances are placed in unique families in the local AstDB for each Asterisk instance. Again, say we have shared family 'foo', and we have two Asterisk instances - ast1 and ast2. ast1 has an EID of 11:11:11:11:11:11, while ast2 has an EID of ff:ff:ff:ff:ff:ff. Say ast1 updates a key in family 'foo':
ast1
/foo/key = bar
ast2 would receive the update for family 'foo', but instead of updating its local copy, it would instead store the value in a new family for ast1 corresponding to its EID:
ast2
/11:11:11:11:11:11/foo/key = bar
If ast2 later updates the same ky in its local AstDB to 'foobar', the received value from ast1 will not be updated. However, ast1 will receive the update, and store the value in a new family for ast2 corresponding to its EID:
ast2
/foo/key = foobar
/11:11:11:11:11:11/foo/key = bar
ast1
/foo/key = bar
/ff:ff:ff:ff:ff:ff/foo/key = foobar
In order to manipulate shared families, two new dialplan functions have been added, DB_SHARED and DB_SHARED_EXISTS. DB_SHARED allows for creation of a shared family, as well as deletion, while DB_SHARED_EXISTS returns whether or not a family is shared:
same => n,Set(DB_SHARED(put,global)=foo) ; share family 'foo' globally
same => n,Set(DB_SHARED(put,unique)=foobar) ; share family 'foobar' uniquely
same => n,NoOp(${DB_SHARED_EXISTS(foo)}) ; returns '1'
same => n,Set(DB_SHARED(delete)=foo) ; remove shared family status for 'foo'
CLI commands were also added to create/delete shared families, and the output of 'database show|showkey' updated to show the shared status of a family/key/value tuple.
Finally, a mechanism for sharing AstDB information was added to the PJSIP stack's res_pjsip_publish_asterisk. This includes a new event type, 'asterisk-db', which contains the values being created/deleted. Necessary configuration parameters were added to the existing configuration objects that support inbound/outbound PUBLISH support. An example of a PUBLISH request with the new event type is shown below:
PUBLISH sip:ast1@127.0.0.1:5061 SIP/2.0
Via: SIP/2.0/UDP 127.0.0.1:5060;rport;branch=z9hG4bKPj1eb182a7-a0aa-4d73-995d-d2ad4b096db2
From: <sip:ast1@127.0.0.1>;tag=7b294642-06ae-4ecf-8637-db8ba2dc4397
To: <sip:ast1@127.0.0.1>
Call-ID: b9463adc-e364-440d-8ce1-842372813b08
CSeq: 48111 PUBLISH
Event: asterisk-db
Expires: 3600
Max-Forwards: 70
User-Agent: Asterisk PBX SVN-mjordan-trunk-astdb-cluster-URL:-r432916M-/trunk
Content-Type: application/json
Content-Length: 156
{"type":"dbstate","eid":"11:11:11:11:11:11","dbstate":{"verb":"put","family":"global_shared","share_type":"global","entries":[{"data":"foo","key":"key1"}]}}
As a note on the power of the frameworks in Asterisk 13 - in this case, both Stasis and PJSIP - the vast bulk of this was written on two plane flights, plus a weekend or so of test writing and cleanup.
[1] http://lists.digium.com/pipermail/asterisk-dev/2015-March/073192.html</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Existing AstDB tests execute successfully, as do the existing PJSIP PUBLISH tests. In addition:
* Unit tests were written (included in this review) that verify the API additions to the AstDB. This includes:
- Verification of creation/delation of shared families
- Verification that creating/deleting an AstDB entry in a shared family publishes the correct Stasis message
- Verification that all keys/values in all shared families can be published via a 'refresh'
* Asterisk Test Suite tests for the PJSIP PUBLISH distribution of the AstDB information are available at: https://reviewboard.asterisk.org/r/4508/</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/trunk/tests/test_db.c <span style="color: grey">(434016)</span></li>
<li>/trunk/res/res_pjsip_pubsub.c <span style="color: grey">(434016)</span></li>
<li>/trunk/res/res_pjsip_publish_asterisk.c <span style="color: grey">(434016)</span></li>
<li>/trunk/res/res_pjsip_outbound_publish.c <span style="color: grey">(434016)</span></li>
<li>/trunk/main/utils.c <span style="color: grey">(434016)</span></li>
<li>/trunk/main/db.c <span style="color: grey">(434016)</span></li>
<li>/trunk/include/asterisk/utils.h <span style="color: grey">(434016)</span></li>
<li>/trunk/include/asterisk/astdb.h <span style="color: grey">(434016)</span></li>
<li>/trunk/funcs/func_db.c <span style="color: grey">(434016)</span></li>
<li>/trunk/configs/samples/pjsip.conf.sample <span style="color: grey">(434016)</span></li>
<li>/trunk/CHANGES <span style="color: grey">(434016)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/4490/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>