[asterisk-dev] Questions about sorcery.

[Matthew Jordan]

On Fri, Jun 21, 2013 at 1:40 AM, Ron Arts <ron.arts at oneip.nl> wrote:

> > If you get *really* interested in playing around with things...
> >
> > There are a number of frameworks that made their way into Asterisk 12 as
> a
> > result of the challenges we found ourselves facing. Two of the big ones
> are:
> > * Stasis - an internal message pub/sub message bus that conveys the
> state of
> > the most important objects throughout Asterisk
> > * Sorcery - a CRUD-ish data abstraction layer that lets you mask away
> real
> > time/conf file parsing/AstDB access/wherever else you want to get your
> data
> >
> > One of the really cool things about Stasis is that - for the primary
> > telephony objects in Asterisk (bridges, channels, device state, etc.) -
> it
> > caches the last known state in an internal cache. Consumers can subscribe
> > for updates; they can also query the cache for the last known value.
> > Currently, all of this is internal, and the cache is rather simple - an
> in
> > memory container of the objects.
> >
> > Sorcery lets you push your run-time objects out to whatever you specify
> (if
> > you have a wizard for it). It can also update objects.
> >
> > Now, this is where it gets really interesting.
> >
> > Technically, with a bit of effort, you could forward Stasis's primary
> caches
> > to a secondary cache that was created with Sorcery. (And no, we haven't
> done
> > this yet - partially because we didn't need to use Sorcery in Stasis, and
> > partially because we've been wanting to keep Stasis as fast as possible.
> > Forwarding things from one cache to another should be relatively fast
> > however) Now you can persist all of the run-time objects in whatever you
> > want - the AstDB - or a MySQL database. Your channel state, bridge state,
> > device state, etc. can all be cached in a hierarchy, going all the way
> out
> > to an external system.
> >
> > Going the other direction - re-creation of state within Asterisk from an
> > external system - would be a monumental amount of effort. However, I
> suspect
> > that simply having Asterisk's state readily available for the perusing
> would
> > open up a lot of interesting applications for people building systems
> with
> > Asterisk.
> >
> > I suspect we're going to have lots of fun things to build on top of
> Asterisk
> > 12.
> >
>
> So basically AMI on steroids. But I would be concerned about performance.
>

Possibly - one of (many) reasons why we haven't gone down this road just
yet. There are some other frameworks in Asterisk 12 to alleviate this
however - we now have generic threadpools that tasks can be serialized and
pushed to. Stasis makes use of these threadpools - so the servicing of
publications does not have to take place on the publisher thread. This
helps prevent situations where publishing data impacts your real-time tasks.


> If asterisk is heavily loaded internal state changes happen really
> often, and pushing
> these out will cost you even more CPU and system load. Being able to
> query the cache
> would only be interesting to re-sync state after a problem, because
> normally you can
> subscribe to changes anyway.


Yes, and CPU/system load is a concern even if you push publication out to a
threadpool. If such a system existed, however, you could:
 * Tell Sorcery to only store items in memory. That would emulate the
system currently. Alternatively, if Sorcery was only populated as a
forwarded caching topic, you could just turn it off all together.
 * Create a configuration mechanism to assign thread priority and/or
affinity to threadpools to prevent them from starving threads servicing
real-time information

This is all hypothetical, of course :-)


> It would be interesting to be able to subscribe
> to stasis externally, but that's what probably stasis-http is for?
>

Sort of.

Stasis-HTTP is going to get renamed to ARI (Asterisk REST Interface), as we
found having multiple "Stasis" things confusing. Stasis is now (mostly)
just the message bus. ARI's purpose is a bit more specialized than Stasis,
although it makes heavy use of Stasis.

ARI exists as a mechanism to control the raw telephony objects in Asterisk
from an external source. When a channel enters into an application that
hands control of the channel over to ARI, that channel is owned by the
external source. The source can manipulate media, create bridges, create
new channels, etc. The best way to describe ARI is it allows you to write
your own applications for Asterisk - rather than relying on VoiceMail, or
Queue, or large monolithic C apps - you can own the channels and bridges
and devices and whatnot and write your own in your language of choice.

This may sound something like AGI - where ARI differs are in a few key
aspects:
 * Where AGI acted in the context of a single channel, with ARI you
implicitly aggregate multiple channels together into application groups.
Manipulating multiple channels with AGI was always a bit tricky and usually
required integration with AMI.
 * With ARI, you can create bridges, channels, and other objects directly
in the context of a specific ARI application. You don't have to Originate a
channel to the dialplan, redirect them into another AGI, then combine them
somehow with a channel already in an AGI if you need a new channel in your
ARI application.
 * You own more than just the channel. Owning the bridges means that you
can bridge channels, remove channels from bridges, combine bridges, etc.
all without involving the dialplan or going back to the dialplan.
 * ARI is asynchronous. Operations can be queued up, stopped, etc. When an
operation isn't in effect, the channel just waits for the next one to come
in. This includes media manipulation.

So, ARI is an interface that uses Stasis and exposes some of Stasis. It
doesn't expose all of it.

You could theoretically build all sorts of things on Stasis - in fact,
David Lee wrote a statsd server integration module as a proof of concept.
There's a brief description of interacting with the message bus on the wiki
at https://wiki.asterisk.org/wiki/display/AST/Using+the+Stasis+Message+Bus

By the way, this may be after the fact, but is there a specific reason
> for choosing
> long-poll http over other message queue protocols like zeromq, or mqtt?
>

ARI uses WebSockets to send events.


>
> I'm sorry if I'm rambling or keeping you from more useful work...
>
>
Not at all! We're excited about the work that's going on in 12 - keep the
questions coming!

-- 
Matthew Jordan
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20130621/73681b0b/attachment-0001.htm>