[asterisk-dev] Asterisk PJSIP stack PUBLISH support

[Matthew Jordan]

Hey everyone!

As you may know, feature freeze is upon us this Wednesday [1], and a
large number of new features and improvements are being made to
Asterisk trunk and put up for code review. A lot of really great work
has been done by the community to help get Asterisk in shape for the
next LTS release (thank-you Corey Farrell!), and I'm confident
Asterisk 13 is going to be one awesome release.

One feature that we've wanted to do for some time, but weren't sure if
we could get done by Wednesday, is the ability for the new SIP stack
to publish event state to other endpoints. This would allow for
Asterisk to use the SIP stack to meet the same kinds of use cases that
the existing clustering mechanisms in Asterisk currently fulfill,
without having to set up an XMPP server or Corosync.

There's a lot of different ways to view this problem, with different
levels of ambition. All of these require a functional publication
framework in the SIP stack, with fully implemented event state
aggregator/compositor functionality. The implications of the
implementation in Asterisk of each of the following scenarios - which
are by no means comprehensive to what all can be done with publish
support - are wildly different:

(1) Allow for Asterisk to publish and consume device state and MWI to
other Asterisk instances. This is probably the “easiest”, as the event
type could be custom to Asterisk and the scope of the problem is
limited to making sure that Asterisk can publish and consume state
from other like Asterisk instances. Since Asterisk has the concept of
externally provided device state/MWI baked into its core, the bulk of
this work has to be done only in the SIP stack and requires little to
no core changes.

(2) Allow for Asterisk to publish/consume presence state (essentially,
extension state) from other Asterisk instances and endpoints. This has
some very interesting applications, as it allows for better Kamailio
integration and for Asterisk to 'farm out' the role of a subscription
server to a SIP proxy. While this is very useful, there's some
problems in the Asterisk core that would have to be ironed out before
this could be supported:

(a) The Stasis message bus has the concept of 'internally generated'
and 'externally generated' updates built into it for all notifications
pushed across it. However, extension state is not pushed across the
message bus. Instead, it composes itself from received device state
updates (which are already aggregated with the notion of
internal/external generation) and notifies extension state consumers
directly without going over the bus again. Thus, unlike the device
states that make up the extension state, it does not, by itself, have
the concept of external/internal composed into it. This makes handling
an externally received extension state problematic: do we update the
extension (if we have a matching one), overriding our internal state?
If so, what happens when one of those device states change again? Or
do we build out a composed state of the hint using our internal
extension state along with the externally provided one?

(b) Unlike device state (which generally has a one-to-one relationship
with something physical), hints that provide extension state are often
generated dynamically using regular expressions in the dialplan. This
"on the fly generation" is very powerful, but provides some
configuration issues: if the configuration to publish this information
is provided in the dialplan, this requires more core changes that may
be channel driver specific. If the configuration to publish is
provided in the channel driver, there must be a sufficiently flexible
mechanism to tie the generated hints to the entities that Asterisk
should publish to.

(3) Allow for Asterisk to publish and consume resource lists (see
Mark's work here [2]). While this is also really cool, there's some
open questions here as well:

(a) When receiving a published resource list event package, if some of
those entities do not map to a known entity on the Asterisk system,
how do we handle it? Do we create them on the fly if possible?

(b) When publishing a resource list, do we publish only those parts of
a resource list that a particular endpoint may care about, or do we
treat the resource list as a whole entity?


With all of this in mind, and given the tight time constraints, we
decided on doing a very 'basic' implementation of publish support
(relatively speaking): we opted for option (1). We didn't want to rush
the implementation of options 2 and 3, and decided that implementing
those features warranted much more discussion on the mailing list or
at AstriDevCon.

Even so, we weren't sure that even option (1) would get done in time –
in which case, we'd put it in a team branch for Asterisk 14. However,
Josh worked some crazy hours and managed to finish up support for a
custom event package distributing MWI and device state between
Asterisk instances, along with improvements to the existing PUBLISH
support in res_pjsip_pubsub. This feature is up for review now at
https://reviewboard.asterisk.org/r/3780/.

This is not to say that this work is done! In fact, we're really just
starting to scratch the surface of Asterisk sharing and consuming
state between other Asterisk instances and other SIP
producers/consumers. Certainly, making Asterisk publish and consume
presence state from other SIP endpoints would be a much more
compelling feature in the future. Beyond that, one can foresee making
Asterisk do all sorts of cool things with different resource lists
composed of presence state, device state, MWI, and whatever else
people want to write a module for. It is a very flexible stack, after
all!

However, we think this is a pretty good first step forward towards
making Asterisk participate in a clustered environment using only SIP.
I think discussing options (2) and (3) – and probably lots more – on
this mailing list and at AstriDevCon for Asterisk 14 would be
fantastic.

Happy reviewing (and impending feature freeze)!

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Versions
[2] https://wiki.asterisk.org/wiki/display/AST/Resource+List+Subscriptions


-- 
Matthew Jordan
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org