<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/2033/1/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations">Component Threading Design for Asynchronous Operations</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~khunt">Ken Hunt</a>
</h4>
<br/>
<h4>Changes (4)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" > <br>|!async_request_chaining.png|border=1!| <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">|*Figure 1. Asynchronous Request Chaining*| <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">|{center}*Figure 1. Asynchronous Request Chaining*{center}| <br></td></tr>
<tr><td class="diff-unchanged" > <br>To support this concept, the Server Component is required to do quite a bit of bookkeeping. When an AMD dispatch is received, the servant must store enough information about the request to forward the processing of the operation to a separate worker thread (managed by the Work Queue object). When it is invoked on the worker thread, the operation will eventually create an AMI callback object to make AMI calls to Component X. When Component X provides its results, the AMI callback object is invoked. That callback object must then cause the remainder of the processing required in the operation to be executed with the appropriate state maintained across all of these thread hops. <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" > <br>|!async_op_state_machine.png|border=1!| <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">|*Figure 2. Server Component Details*| <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">|{center}*Figure 2. Server Component Details*{center}| <br></td></tr>
<tr><td class="diff-unchanged" > <br>This process (i.e. execute current state on Work Queue, invoke AMI call during transition to new state, AMI callback reschedules execution on Work Queue) can be repeated as many times as required. For each AMI call required, an additional state is defined in the state machine. Exceptions to this would be when a collector for multiple similar AMI calls is used to aggregate the results of multiple outbound calls into a single response. (Example: collect the endpoints for each Session in a given set.) Such aggregate operations can be represented as a single state, even though multiple AMI invocations are made, by using a shared collector in the AMI callbacks, which is ultimately responsible for the reschedule step. <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-Overview"></a>Overview</h3>
<p>This page describes an approach to supporting asynchronous operations in a component, as currently implemented in the Basic Routing Service. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-Background"></a>Background</h3>
<p>Asterisk SCF components can make use of Ice's <a href="http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/Cpps.9.8.html" class="external-link" rel="nofollow">Asynchronous Message Dispatch (AMD)</a>, a server-side technique for scaling the number of requests a server can handle. Using AMD a server can receive a request but then suspend its processing in order to release the dispatch thread, and execute the request on another thread. Components may also use the client-side equivalent, <a href="http://www.zeroc.com/doc/Ice-3.4.1-IceTouch/manual/Cpp.7.15.html" class="external-link" rel="nofollow">Asynchronous Method Invocation (AMI)</a>, a way to invoke an operation on a server which never blocks the calling thread even for two-way calls. However, many (if not most) Asterisk SCF components will act as both client and server, where an incoming request will involve invoking one or more operations on other components in order to achieve a result. Even operations which return void and have no "out" parameters may throw exceptions, which would need to be communicated to the initiator of the request for a two-way operation. </p>
<p>The Basic Routing Service falls into the category of components which will act as both server and client within a given servant. An approach for managing the lifecycle of a server operation was implemented in the routing service, and is described on the remainder of this page. The impact of the design on the component's state replication is also discussed. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-Design"></a>Design</h3>
<p>The approach implemented in the Basic Routing Service employs what the Ice documentation refers to as <em>asynchronous request chaining</em>, where an incoming AMD request is processed, and in that processing executes an AMI request on some other component. This simple concept is shown in Figure 1, where we see a client invoke an operation ("op") on the Server Component. The Server Component dispatches dependent operations to external Component X, but remains free to process other work in the intervening time from when a reply is sent from Component X. When Component X services the request, the Server Component is able to complete the processing of the original client request, and the AMD callback object is used to provide results to the client. </p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<td class='confluenceTd'><span class="image-wrap" style=""><img src="/wiki/download/attachments/10649930/async_request_chaining.png?version=2&modificationDate=1293136023472" style="border: 1px solid black" /></span></td>
</tr>
<tr>
<td class='confluenceTd'><div class="" align="center"><b>Figure 1. Asynchronous Request Chaining</b></div>
</td>
</tr>
</tbody></table>
</div>
<p>To support this concept, the Server Component is required to do quite a bit of bookkeeping. When an AMD dispatch is received, the servant must store enough information about the request to forward the processing of the operation to a separate worker thread (managed by the Work Queue object). When it is invoked on the worker thread, the operation will eventually create an AMI callback object to make AMI calls to Component X. When Component X provides its results, the AMI callback object is invoked. That callback object must then cause the remainder of the processing required in the operation to be executed with the appropriate state maintained across all of these thread hops. </p>
<p>The solution used in the Basic Routing Service was to implement an operation-specific state machine. Figure 2 shows the creation of this state machine at operation 1.3. As can be inferred from the diagram, the object that implements the state machine is also capable of being enqueued to a Work Queue. </p>
<p>The Operation State Machine is designed to have a state handler for each state. This handler is maintained internally in a boost::function<void ()> reference. Whenever the operation processing requires an AMI call to another component, a callback operation is created to receive the results. This is considered a state transition point. The operation transitions its state internally, and invokes the remote call on Component X. Until the results of the AMI call are received, there is no thread blocked within the Server Component. The AMI callback will reschedule the operation to be processed on the work queue, which re-enters the state machine in the state last transitioned to prior to the AMI call. </p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<td class='confluenceTd'><span class="image-wrap" style=""><img src="/wiki/download/attachments/10649930/async_op_state_machine.png?version=1&modificationDate=1293125402006" style="border: 1px solid black" /></span></td>
</tr>
<tr>
<td class='confluenceTd'><div class="" align="center"><b>Figure 2. Server Component Details</b></div>
</td>
</tr>
</tbody></table>
</div>
<p>This process (i.e. execute current state on Work Queue, invoke AMI call during transition to new state, AMI callback reschedules execution on Work Queue) can be repeated as many times as required. For each AMI call required, an additional state is defined in the state machine. Exceptions to this would be when a collector for multiple similar AMI calls is used to aggregate the results of multiple outbound calls into a single response. (Example: collect the endpoints for each Session in a given set.) Such aggregate operations can be represented as a single state, even though multiple AMI invocations are made, by using a shared collector in the AMI callbacks, which is ultimately responsible for the reschedule step.</p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-ImplementationDetails"></a>Implementation Details</h3>
<h4><a name="ComponentThreadingDesignforAsynchronousOperations-Sharedpointers"></a>Shared pointers</h4>
<p>The Work Queue only accepts shared pointers to Work objects. This ensures that the Work object is always a valid object as it is called from the Work Queue's internal thread(s). However, this also makes it difficult for the Operation State Machine (or a created AMI callback object) to reschedule an operation, since we don't want an object to have an internal shared pointer to itself. The designed solution is for all Operation State Machines to require a reference to an object that implements a Operation Manager interface. The Operation Manager maintains a collection of shared pointers to all ongoing operations. It provides a finished() operation so that each Operation can report when it is complete, and a reschedule() operation so that a raw pointer to "this" can be mapped to it's shared pointer, which can then be enqueued to the Work Queue. The Operation Manager could also be instrumented in the future to provide runtime performance metrics such as number of ongoing operations, total operations processed, average time to complete, etc. </p>
<h4><a name="ComponentThreadingDesignforAsynchronousOperations-WorkQueuedetails"></a>Work Queue details</h4>
<p>The Work Queue is implemented as a pure interface in WorkQueue.h, and then implemented as SimpleWorkQueue.h/.cpp. The main entry class of the Basic Routing Service creates the SimpleWorkQueue, and passes only the abstract interface to the Session Router servant implementation. The component could easily be made configurable as to the specific type of WorkQueue implementation to use. </p>
<p>The Work Queue enqueue() operation returns an abstract class called PoolId. An implementation that supports a thread pool would return a token that identifies the particular thread that the Work was enqueued to, in the event that the Worker object needs to schedule other tasks on the same thread that it is running on. The Routing Service has no such requirements at this time. </p>
<p>I chose to use a named method as the entry point from the Work Queue, as I think it is generally more understandable than overloading the "()" operator. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-ThoughtsonStateReplication"></a>Thoughts on State Replication</h3>
<p>The current state machine implementation is just a trivial function reference to execute the current state. That was all that was needed to support asynchronous processing for the moment. However, tracking specific states with more detail will be useful when adding state replication support. In particular, additional states should be added to indicate when an operation is waiting for a callback object to reschedule it. This is currently not done... only states that require a state handler to be invoked from the WorkQueue are implemented. The "waiting on callback" state is simply implicit, and there is no execution of the operation until the callback does so. Implementing a real state machine (i.e. more than just a function reference to handle execution in the current state) will make serialization of the state simpler. </p>
<p>Most state machines I've implemented in the past had mechanisms for other component to "listen" for state transitions. The internal object which pushes updates to the State Replicator would likely be such a listener. This all requires a bit more "proper" State Machine implementation, but it seems like it will provide useful functionality, and is certainly not a technically challenging thing to add. </p>
<h3><a name="ComponentThreadingDesignforAsynchronousOperations-SpecificElementsoftheBasicRoutingServiceImplementation"></a>Specific Elements of the Basic Routing Service Implementation</h3>
<p>The Routing Service (primarily) provides servants for interfaces from two different namespaces, namely AsteriskSCF.Core.Routing.LocatorRegistry, and AsteriskSCF.SessionCommunications.SessionRouter. The current implementation provides AMD implementations for all the operations in these two interfaces. The LocatorRegistry.lookup() operation doesn't use the Operation State Machine mechanisms described here, but rather does a fairly simple AMD -> AMI chaining by embedding the AMD callback into a custom AMI callback that collects the results of querying all of the remote Endpoint Locators that have a regular-expression match to the destination being sought. (The collection functionality is probably a bit of overkill, since we would hope that only one Endpoint Locator has a regex match for the destination. And the current implementation just picks one if there happen to be multiple matches. But there is a place to put logic for deconflicting/choosing when we need to, and we will certainly need to be deliberate in how we resolve such a scenario.) </p>
<p>The SessionRouter operations all use the mechanisms described here. Two of the operations actually require the lookup() operation of the LocatorRegistry, which is now AMD (-> AMI, as described above). So our callback mechanism is a specialization of the lookup() operation's AMD callback object. The local call executes just as an actual remote call into lookup() would, but the responses from lookup() to the AMD callback just happen to stay local. </p>
<p>Other external operations (to the Session and Bridge Manager, for instance) are still executed using synchronous calls. Of course they are being executed on a dedicated Worker thread. Also, if benchmarking or other factors drive us to using AMI to operation on the Session or Bridge Manager, it will be fairly simple to extend the current implementation by adding additional state handlers. Once an operation is implemented as an Operation State Machine, the infrastructure is all in place to make such changes fairly easy to do. </p>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;">
<a href="https://wiki.asterisk.org/wiki/users/viewnotifications.action" class="grey">Change Notification Preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=10649930&revisedVersion=17&originalVersion=16">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>