<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/2033/1/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations">Component Threading Design for Asynchronous Operations</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~khunt">Ken Hunt</a>
</h4>
<br/>
<h4>Changes (1)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" > <br>h3. Background <br></td></tr>
<tr><td class="diff-changed-lines" >Asterisk SCF components can make use of Ice's [Asynchronous Message Dispatch (AMD)|http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/Cpps.9.8.html], a server-side technique for <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">increasing scalability.</span> <span class="diff-added-words"style="background-color: #dfd;">scaling the number of requests a server can handle.</span> Using AMD a server can receive a request but then suspend its processing in order to release the dispatch <span class="diff-added-words"style="background-color: #dfd;">thread, and execute the request on another</span> thread. Components may also use the client-side equivalent, [Asynchronous Method Invocation (AMI)|http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/Cpp.7.15.html], a way to invoke an operation on a server which never blocks the calling thread even for two-way calls. However, many (if not most) Asterisk SCF components will act as both client and server, where an incoming request will involve invoking one or more operations on other components in order to achieve a result. Even operations which return void and have no "out" parameters may throw exceptions, which would need to be communicated to the initiator of the request for a two-way operation. <br></td></tr>
<tr><td class="diff-unchanged" > <br>The Basic Routing Service falls into the category of components which will act as both server and client within a given servant. An approach for managing the lifecycle of a server operation was implemented in the routing service, and is described on the remainder of this page. The impact of the design on the component's state replication is also discussed. <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<h2><a name="ComponentThreadingDesignforAsynchronousOperations-%3CIncompleteDraft%21%21%21%3E"></a><b><Incomplete Draft!!!></b></h2>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-Overview"></a>Overview</h3>
<p>This page describes an approach to supporting asynchronous operations in a component, as currently implemented in the Basic Routing Service. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-Background"></a>Background</h3>
<p>Asterisk SCF components can make use of Ice's <a href="http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/Cpps.9.8.html" class="external-link" rel="nofollow">Asynchronous Message Dispatch (AMD)</a>, a server-side technique for scaling the number of requests a server can handle. Using AMD a server can receive a request but then suspend its processing in order to release the dispatch thread, and execute the request on another thread. Components may also use the client-side equivalent, <a href="http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/Cpp.7.15.html" class="external-link" rel="nofollow">Asynchronous Method Invocation (AMI)</a>, a way to invoke an operation on a server which never blocks the calling thread even for two-way calls. However, many (if not most) Asterisk SCF components will act as both client and server, where an incoming request will involve invoking one or more operations on other components in order to achieve a result. Even operations which return void and have no "out" parameters may throw exceptions, which would need to be communicated to the initiator of the request for a two-way operation. </p>
<p>The Basic Routing Service falls into the category of components which will act as both server and client within a given servant. An approach for managing the lifecycle of a server operation was implemented in the routing service, and is described on the remainder of this page. The impact of the design on the component's state replication is also discussed. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-Design"></a>Design</h3>
<p>The approach implemented in the Basic Routing Service employs what the Ice documentation refers to as <em>asynchronous request chaining</em>, where an incoming AMD request is processed, and in that processing executes an AMI request on some other component. This simple concept is shown in Figure 1. A client invoke some operation ("op") on the Server Component. The Server dispatches dependent operations to external Component X, but is free to process other work in the intervening time from when a reply is sent from Component X. </p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<td class='confluenceTd'><span class="image-wrap" style=""><img src="/wiki/download/attachments/10649930/async_request_chaining.png?version=1&modificationDate=1293125396126" style="border: 1px solid black" /></span></td>
</tr>
<tr>
<td class='confluenceTd'><b>Figure 1. Asynchronous Request Chaining</b></td>
</tr>
</tbody></table>
</div>
<p>To support this concept, the Server Component is required to do quite a bit of bookkeeping. When an AMD dispatch is received, the servant must store enough information about the request to forward the processing of the operation to a separate worker thread. When it is invoked on the worker the thread, the operation will eventually create an AMI callback object to make AMI calls to Component X. When Component X provides its results, the AMI callback object is invoked. That callback object must then cause the remainder of the processing required in the operation to be executed with the appropriate state maintained across all of these thread hops. </p>
<p>The solution used in the Basic Routing Service was to implement an operation-specific state machine. Figure 2 shows the creation of this state machine at operation 1.3. As can be inferred from the diagram, the object that implements the state machine is also capable of being enqueued to a Work Queue. </p>
<p>The Operation State Machine is designed to have a state handler for each state. This handler is maintained internally in a boost::function<void ()> reference. Whenever the operation processing requires an AMI call to another component, a callback operation is created to receive the results. This is considered a state transition point. The operation transitions its state internally, and invokes the remote call on Component X. Until the results of the AMI call are received, there is no thread blocked within the Server Component. The AMI callback will reschedule the operation to be processed on the work queue, which enters the state machine in the state transitioned to prior to the AMI call. </p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<td class='confluenceTd'><span class="image-wrap" style=""><img src="/wiki/download/attachments/10649930/async_op_state_machine.png?version=1&modificationDate=1293125402006" style="border: 1px solid black" /></span></td>
</tr>
<tr>
<td class='confluenceTd'><b>Figure 2. Server Component Details</b></td>
</tr>
</tbody></table>
</div>
<p>This process can be repeated as many times as required. For each AMI call required, an additional state is defined in the state machine. Exceptions to this would be when a collector for multiple similar AMI calls is used to aggregate the results of multiple outbound calls into a single response. </p>
<p><TBD... rest is placeholder...></p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-ImplementationDetails"></a>Implementation Details</h3>
<h4><a name="ComponentThreadingDesignforAsynchronousOperations-Sharedpointers"></a>Shared pointers</h4>
<p>The Work Queue only accepts shared pointers to Work objects. This ensures that the Work object is always a valid object as it is called from the Work Queue's internal thread(s). This makes it difficult for the State Machine (or a created AMI callback object) to reschedule an operation, since we don't want an object to have an internal shared pointer to itself. The designed solution is for all Operation State Machines to require a reference to an object that implements a Operation Manager interface. The Operation Manager maintains a collection of shared pointers to all ongoing operations. It provides a finished() operation so that the Operation can indicated when it is complete, and a reschedule() operation so that a raw pointer to "this" can be mapped to it's shared pointer, which can then be enqueued to the Work Queue. The Operation Manager could also be instrumented to provide runtime information, such as number of ongoing operations, total operations processed, etc. </p>
<h4><a name="ComponentThreadingDesignforAsynchronousOperations-WorkQueuedetails"></a>Work Queue details</h4>
<p>The Work Queue is implemented as a pure interface in WorkQueue.h, and then implemented as SimpleWorkQueue.h/.cpp. The main entry class of the Basic Routing Service creates the SimpleWorkQueue, and passes only the abstract interface to the Session Router servant implementation. The component could easily be made configurable as to the specific type of WorkQueue implementation to use. </p>
<p>The Work Queue enqueue() operation returns an abstract class called PoolId. An implementation that supports a thread pool would return a token that identifies the particular thread that was enqueued to, in the event that the Worker object needs to schedule other tasks on the same thread that it is running on. The Routing Service has no such requirements at this time. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-ThoughtsonStateReplication"></a>Thoughts on State Replication</h3>
<p>The current state machine implementation is just a trivial function reference to execute the current state. That was all that was needed to support asynchronous processing for the moment. However, tracking specific states with more detail will be useful when adding state replication support. In fact, additional states should be added to indicate when an operation is waiting for a callback object to reschedule it. This is currently not done... only states that require a state handler to be invoked from the WorkQueue are implemented. The "waiting on callback" state is simply implicit, and there is no execution of the operation until the callback does so. Tracking the current state in more than just a function reference will make serialization of the state simpler. </p>
<p>Most state machines I've implemented in the past had mechanisms for other component to "listen" for state transitions. The internal object which pushes updates to the State Replicator would likely be such a listener. This all requires a bit more "proper" State Machine implementation, but it seems like it will provide useful functionality, and is certainly not a technically challenging thing to add. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-SpecificElementsoftheBasicRoutingServiceImplementation"></a>Specific Elements of the Basic Routing Service Implementation</h3>
<p>The Routing Service (primarily) provides servants for interfaces from two different namespaces, namely AsteriskSCF.Core.Routing.LocatorRegistry, and AsteriskSCF.SessionCommunications.SessionRouter. The implementation provides AMD implementations for all the operations in these two interfaces. The LocatorRegistry.lookup() operation doesn't use the Operation State Machine mechanisms described here, but rather does a fairly simple AMD -> AMI chaining by embedding the AMD callback into a custom AMI callback that collects the results of querying all of the remote Endpoint Locators that have a regular-expression match to the destination being sought. (The collection functionality is probably a bit of overkill, since we would hope that only one Endpoint Locator has a regex match for the destination. And the current implementation just picks one if there happen to be multiple matches. But there is a place to put logic for deconflicting/choosing when we need to.) </p>
<p>The SessionRouter operations all use the mechanisms described here. Two of the operations actually require the lookup() operation of the LocatorRegistry, which is now AMD (-> AMI, as described above). So our callback mechanism is a specialization of the lookup() operation's AMD callback object. The local call executes just as an actual remote call into lookup() would, but the AMD callback just happens to stay local. </p>
<p>Other external operations (to the Session and Bridge Manager, for instance) are still executed using synchronous calls. Of course they are being executed on a dedicated Worker thread. Also, if benchmarking or other factors drive us to using AMI to operation on the Session or Bridge Manager, it will be fairly simple to extend the current implementation by adding additional state handlers. Once an operation is implemented as an Operation State Machine, the infrastructure is all in place to make such changes fairly easy to do. </p>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;">
<a href="https://wiki.asterisk.org/wiki/users/viewnotifications.action" class="grey">Change Notification Preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=10649930&revisedVersion=9&originalVersion=8">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>