[asterisk-dev] Proposal to bring pjproject back into the fold

Wed Jan 20 09:49:32 CST 2016

On Wed, Jan 20, 2016 at 8:22 AM, Jared Smith <jaredsmith at jaredsmith.net>
wrote:

>
> On Tue, Jan 19, 2016 at 1:04 PM, George Joseph <
> george.joseph at fairview5.com> wrote:
>
>> If we don't static link, we'll never know what's going on.
>
>
>
> I'm going to push back on this as well.  Short of going the route of
> static linking, what can we do to improve the current situation with
> dynamic linking?  Thoughts?  Ideas?  Improved testing?  More robust code?
>

I think we're simply facing a fundamental difference in how pjproject
expects to be used, versus how we're attempting to use it.

Generally, when building pjproject, developers building pjproject must find
the right invocation that addresses:
(1) Their unique system configuration and its dependencies. Namely, this is
telling pjproject which third party packages/libraries it should not use
(more on this in a bit as well).
(2) Values for #defined parameters that must be set in pjproject for their
Asterisk system. This is usually done via either:
(2a) A unique site configuration header file
(2b) Parameters passed to ./configure directly that determine the value of
these directives.

Needless to say, getting these invocations correct the first time is
challenging. Often it is not obvious when you get them wrong. For example,
failing to set the correct maximum size of your SIP packets in pjproject
will simply result in Asterisk rejecting those SIP messages that exceed the
max packet size; this has serious implications for anyone wanting to handle
SIP packets that contain many ICE candidates.

This has resulted in an extremely frustrating situation for both Asterisk
users and developers. The Asterisk wiki page that details how to configure
and build pjproject [1] has been the result, but the complexity of that
page highlights just how bad this situation has gotten.

So why is pjproject such a pain?

There's probably a number of reasons, and I'm probably forgetting some, but
I'd boil the top ones down to the following:

* pjproject embeds third party libraries.

Some of those third party libraries are also libraries that Asterisk can
use; if Asterisk needs to use them, you have to tell pjproject to not use
its embedded version. While it is easy to say that pjproject should "just
not do that", that misses the original - and still one of the most common -
use cases of pjproject; namely, as an embedded systems library. For people
building SIP devices on embedded platforms, having embedded versions of
GSM, SRTP, and other useful libraries is actually advantageous. Packages
are not usually available for, say, VxWorks, and if I know pjproject can be
built using my esoteric toolchain, I don't have to try and make libsrtp
build as well with it.

As a result, pjproject is highly unlikely to ever remove their embedding of
third party libraries, despite our attempts at convincing them to do that.
They serve *more* than just the Asterisk project, and our needs are not
always going to match 100% with theirs.

The implications of that are that we're going to need to work around and/or
deal with the third party libraries they embed in some fashion.

* pjproject uses defined constants for certain key constraints

This includes whether or not it supports IPv6, the maximum size of SIP
packets, debug assertions, and other items. When pjproject is configured
with one set of values, and Asterisk is built against a different set of
constraints, very, very, very bad things can happen.

One work around for this would be to patch pjproject to provide accessors
to its defined constants. Asterisk would then use those accessors to get at
the values, as opposed to using the constants directly. While that
substantially alleviates the issue, the discontinuity between what
pjproject views as reality and what Asterisk views isn't uniquely solved by
this (we would just stop crashing).

As an example, consider a case where a package builds pjproject with a
maximum SIP packet length of 8192. That means my package is never going to
be useful for anyone wanting Asterisk to handle WebRTC SDPs. Likewise, if
I've built pjproject with a maximum SIP packet length of 16k, I may have
just made it so that my package cannot be used on embedded devices using
that distribution of Linux.

Could this be a configuration driven parameter, as opposed to a defined
constant? Maybe, although that isn't pjproject's model. Could a patch be
written that makes it do that? Yes, but there are a large number of these
types of options, and the burden that this continues to put on the Asterisk
developer community is high.

* pjproject static/shared libraries is confusing for many people

It is not uncommon for people to build static libraries of pjproject the
first time around (missing the instructions to not do that), install them,
and run Asterisk. Asterisk then will crash or do something terrible. They
will then attempt to install the shared object libraries, not removing the
static libraries. Asterisk will still run, and now crash in a different
fashion. They will then attempt to remove the static libraries, and now
Asterisk will complain that it can't find pjproject because you're using
RHEL and your pkg-config path isn't set up correctly.

Is this all user error? Of course. But this happens *a lot*, and it is
signals that there is a problem with the environment that we're expecting
people to operate under.

Can all of the above issues be alleviated in some fashion via various
technical changes? I'd hazard a guess and say "yes". However, I think that
even if we alleviate them, we're going to still be left with a situation
where the most important functionality in Asterisk - its SIP stack - is
more challenging to get configured and set up than what users feel it
should be. That's a bad situation for users. I know the packagers feel that
having anything embedded or statically linked is worse for users;
unfortunately, after witnessing where we're at after nearly three years
with the current model, I disagree with that feeling.

I think a number of developers in the Asterisk project are frustrated by
this situation. There has been a lot of time and energy invested into
trying to make the current model work, and we still see a lot of users
frustrated by the current requirements we're levying on them. Continuing to
throw more development time at a problem to satisfy a small group of people
who categorically refuse to brook any compromise is a difficult proposition
for me.

I'll go so far as to say that in my time on the Asterisk project, I
personally feel that the single worst technical decision I've made was in
making it so that the only way to use pjproject was via shared object
libraries. The pain to users, and the impact this has had on adoption of
the PJSIP stack in Asterisk, isn't something I can even begin to calculate.

[1]
https://wiki.asterisk.org/wiki/display/AST/Building+and+Installing+pjproject

-- 
Matthew Jordan
Digium, Inc. | Director of Technology
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20160120/df37e538/attachment-0001.html>