[asterisk-dev] Asterisk 14 Status and Project Timeline

Mon May 4 13:14:23 CDT 2015

Hello everyone!

It's May! As you may know, the feature freeze announcement for
Asterisk 14 is scheduled for the third Wednesday in May, which means
that it is coming up on us awfully fast [1]. Since the overall
Asterisk 14 deadline is not that far behind the announcement – a
little less than three months away from today – I thought it might be
good to review the status of Asterisk 14, and make some
recommendations about the timeline for the release of Asterisk 14.

To recap, at AstriDevCon last year [2], the community proposed three
broad areas to focus on:

1) Move the project to Git, and improve the project infrastructure to
make it clearer how someone contributes to Asterisk.

2) Improve documentation, particularly for scenarios. Make it easier
to configure and deploy Asterisk.

3) Continue to make Asterisk a better media application server by
focusing on improvements to ARI and scalability issues.

The following is an update on the progress the community has made in
each of these broad areas.

= Moving the project to Git =

How the saga has unfolded:

Back in September [3], an initial investigation was done on various
platforms that the project could use. After some good discussion,
Gerrit – along with Zuul and Jenkins – was proposed as a possible
solution. Paul Belanger and others provided some test infrastructure,
and a number of people in the project evaluated it as a possible
solution. More discussion ensued [4]. Finally, after a lot of behind
the scenes wrangling of infrastructure (including, but not limited to,
Crowd, Crowd OpenID, Gerrit, Zuul, Jenkins, and a host of plugins), a
public Gerrit server was stood up [5], the Testsuite was moved over
[6], and a test Asterisk repo was made for people to play around with
[7]. Once we felt mildly comfortable with the whole setup, the entire
Asterisk project was moved over [8].

We've been live on Gerrit now for three weeks, and – while there are
still some glitches to work out – I think the overall result has been
a net positive for the project.

Of course, there is still work to do:

1) Documentation – while it exists and is sufficient to get someone
working in Gerrit [9] – could be better still. In particular, the
general flow of a new developer coming to
asterisk.org/wiki.asterisk.org and finding the right process for
contributing a patch is a bit confusing. That needs to get cleaned up
a bit more before we really “market” the change over to Git.

2) The largest remaining infrastructure problem is getting Zuul and
Jenkins fully working. While both are deployed, Zuul is exhibiting a
lot of SSH timeouts with the Gerrit server – which not even an upgrade
of Gerrit resolved. Luckily, the Gerrit bot in #asterisk-commits
(CaptainPants) uses the same underlying library as Zuul to monitor the
Gerrit event stream, and it's working just fine. Presumably, one could
assume that there exists a path forward on getting Zuul fixed as well.

3) The repotools repository is still being migrated to allow us to
make releases from the Asterisk repository. I expect the final patches
for this to be up on Gerrit by the end of the week – which should put
us in position to make 11.18.0 and 13.4.0 next week. (The first
release from Git! Huzzah!)

4) There are a number of small issues pointed out by Corey, George,
and others that need to get tweaked in Gerrit. These are slowing
getting worked as people (mostly myself) have time to get to them.

= Documentation Improvements =

Over the past few months, we have slowly but surely made progress on
improving the documentation in Asterisk and on wiki.asterisk.org. This
includes:

* Reorganizing the documentation and improving many of the existing wiki pages.

* Providing in depth documentation on the ARI interface [10] and
providing sample ARI applications – mostly in node.js. Sample
applications include:
 - A suite of simple tutorial applications that match the
documentation on the wiki [11]
 - A very complete basic voice mail application [12]
 - A conferencing application, with many of the features of ConfBridge [13]

* Adding more PJSIP examples and in-depth information on various
scenarios. Examples include how to handle Remote Attended Transfers
with PJSIP [14], how transport selection works in the Asterisk PJSIP
stack [15], and examples of using the PJSIP configuration wizard to
ease configuration of a system [16].

In addition, a very large documentation effort was made to provide a
set of sample configs that represent a “basic PBX.” These
configuration files represent an SMB PBX for a fictitious company, and
provide a starting point for people looking to configure Asterisk in a
similar scenario. The configuration files exist in the
configs/basic-pbx folder in Asterisk, and – once the documentation on
the wiki is enhanced a bit more - will support their own make target
as well. Ideally, in the future, additional sets of configuration
files can be provided for other scenarios. This will help to
demonstrate some of the Asterisk best practices for a variety of
scenarios that have not always been well documented.

Of course, with documentation in particular, the goal is never quite achieved.

First, there is still a lot of documenting to be done on ARI. A number
of more complex scenarios are not well covered by the documentation on
the wiki, including the usage of Snoop channels and ARI/dialplan
interaction. For ARI to continue to improve – and be useful – we need
to continue to improve the documentation of that API.

Second, while the PJSIP reference documentation is arguably more
complete than chan_sip's ever has been – thanks to the enforcement of
configuration option documentation in Sorcery and the Configuration
Framework – examples of how to configure PJSIP for a variety of
scenarios could still use improvement. We still see a lot of people on
the asterisk-users list working through scenarios that need more
clarification. Probably the most difficult aspect of PJSIP is simply
getting the pjproject libraries built and installed correctly. For the
PJSIP stack in Asterisk to really mature, the barrier for entry needs
to be lowered as much as possible.

Finally, while the basic-pbx set of sample configuration files is a
great starting point, a lot of work needs to be done on the wiki to
explain those files. This naturally should feed back into the overall
documentation efforts, such that the documentation on the wiki and the
sample configuration files in Asterisk are not disjoint.

= ARI Improvements =

Over the past few months, some progress has been made on a variety of
ARI improvements. Some highlights include:

* In 13.2.0, a “who created me” parameter was added to the POST
/channels operation. This is needed for a lot of real-world scenarios,
in which various systems need the accountcode and linkedid information
associated with the 'calling' channel to be propagated over to the
'callee' channel. Allowing users to specify which channel caused a
channel to be created allowed a number of existing billing systems to
make sense of the operations occurring within ARI.

* Also in 13.2.0, the ability to handle priority labels in the
dialplan was added to both the POST /channels operation as well as the
continue operation. This made integration of ARI with existing
dialplan easier (and made it easier to know where you're releasing
channels to).

* In 13.3.0, a 'redirect' option was added to the channels resource
that mimics that Transfer dialplan application. This more easily
allows for scenarios where calls need to be moved out of an ARI
application to some other system. Note that this is similar to what
Paul Belanger demoed at the AstriCon hackathon, where he was able to
effectively move a live, bridged call from one Asterisk server over to
another Asterisk server. Previously, Paul had to make use of a
combination of the continue operation and the Transfer dialplan
application; this enhancement lets you do the operation completely
within ARI.

* Coming in 13.4.0, the ability to intercept channel hold has been
added. Rather than putting the other party in a bridge on hold, this
will instead raise an ARI event indicating that a channel wanted to
put the other party on hold. This allows an ARI application to
customize the behavior in the application when someone hits the hold
button on their phone – which is nice if you are trying to write your
own SLA application.

While all of these enhancements are good and interesting, they don't
address some of the larger projects outlined at AstriDevCon. In fact,
since AstriDevCon, an even larger ARI wish list has been put together
by the community [17]. A little bit of progress has been made on those
items; but to date, I would guess that we are still a long ways away
from seeing the projects on the Asterisk 14 roadmap from coming to
fruition [18].

Why is that the case?

Well, not surprisingly, given the scope of Asterisk 12 and Asterisk
13, there's been a few bugs filed. So a large number of folks in the
community have been trying to make sure those got knocked out in a
timely fashion – and so far, we've (more or less) been able to hit the
most important and critical bugs. On top of that, the Git migration
took a lot of time and energy from a number of people on the team here
at Digium – and that also limited who was available to work on new
enhancements for ARI. Lastly, of course, is that no one from the
community has decided to write a patch for the unimplemented yet
requested ARI features. Working code, more than anything, is what
drives an open source project. While all of the requested features are
great ideas, they are nothing more than requests until someone decides
to do the actual work. So, if you see a feature on the list and you'd
like to make sure it happens for Asterisk 14 but aren't sure where to
start, jump in #asterisk-dev or on this mailing list and we would be
happy to help you out.

= Scalability =

While focusing on scalability was not as high on the list as the
aforementioned issues, it has been a topic of conversation on the
mailing list a few times [19] [20]. Much of this has centered around
better integration between Asterisk and Kamailio. However, with the
exception of Daniel's proposal for a new authentication scheme, I
think we're still a little muddy on where the real problem areas with
scaling Asterisk horizontally lie. In many conversations – both on the
asterisk-dev list and in person with various people in the community –
I continually get the impression that while there are many areas for
improvement, no one is really quite sure what the real, intransigent
issues are, much less how to resolve them.

To that extent, I'd like to resurrect some of the threads that were
going on a few months ago, and try to reach some conclusion about what
areas in Asterisk are best to focus on to solve some of the
scalability issues people have alluded to over the years. Time is
always a limited commodity – and having people in the community
thinking about the most important problems will give Asterisk the
biggest payoff when features targeting scalability issues are
implemented.

= Conclusions =

So where does this leave us?

While we've made a lot of great progress on a number of the items from
AstriDevCon, it feels like we're still quite a ways out from the goals
outlined in that meeting. We still have a lot of infrastructure
improvements to make, we have a lot of features to document and
existing documentation to improve on, we have a plethora of ARI
features that people would love to have, and we're still not quite
sure where we should improve Asterisk to make it scale better.

All of that is a lot to cram in the next few months, and cramming
things is rarely a good idea.

To that extent, I'm proposing that we hold off on the Asterisk 14
deadlines for now. Having a stable release schedule is great for the
project and for users, but not at the expense of having a release that
doesn't meet its goals. While I don't want to hold users hostage to an
unimplemented feature list, I also want to make sure that we take
advantage of the Standard release, and that whatever we release moves
the Asterisk project substantially forward.

If you have any thoughts or comments on the above, I'd love to hear them.

And, as always, thanks for supporting the Asterisk project!

Matt

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Versions
[2] https://wiki.asterisk.org/wiki/display/AST/AstriDevCon+2014
[3] http://lists.digium.com/pipermail/asterisk-dev/2014-September/070303.html
[4] http://lists.digium.com/pipermail/asterisk-dev/2014-December/072024.html
[5] https://gerrit.asterisk.org
[6] http://lists.digium.com/pipermail/asterisk-dev/2015-March/073461.html
[7] http://lists.digium.com/pipermail/asterisk-dev/2015-April/074129.html
[8] http://lists.digium.com/pipermail/asterisk-dev/2015-April/074589.html
[9] https://wiki.asterisk.org/wiki/display/AST/Gerrit+Usage
[10] https://wiki.asterisk.org/wiki/pages/viewpage.action?pageId=29395573
[11] https://github.com/asterisk/ari-examples
[12] https://github.com/asterisk/voicemail-docs
[13] https://github.com/asterisk/node-confbridge
[14] https://wiki.asterisk.org/wiki/display/AST/res_pjsip+Remote+Attended+Transfers
[15] https://wiki.asterisk.org/wiki/display/AST/PJSIP+Transport+Selection
[16] https://wiki.asterisk.org/wiki/display/AST/PJSIP+Configuration+Wizard
[17] https://wiki.asterisk.org/wiki/display/AST/ARI+Feature+Wish-list
[18] https://wiki.asterisk.org/wiki/display/AST/Asterisk+14+Projects
[19] http://lists.digium.com/pipermail/asterisk-dev/2015-February/072748.html
[20] http://lists.digium.com/pipermail/asterisk-dev/2015-March/073217.html

-- 
Matthew Jordan
Digium, Inc. | Director of Technology
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org