[asterisk-dev] Recommendations for using a SIP stack with Asterisk

[Mark Michelson]

On 11/09/2012 06:19 PM, Faidon Liambotis wrote:
> On 11/09/12 23:28, Mark Michelson wrote:
>> 1. If we use PJSIP, no distros have packages for it. If we use
>> ReSIProcate, then distro package support is limited. We cannot, as an
>> alternative to bundling it with Asterisk, tell people to use their
>> distro's package.
> That's fixable, isn't it? If you use a sensible library, then distros
> will happily package that and the whole ecosystem will profit. Note that
> both resiprocate and sofia-sip are packaged e.g. in Debian/Ubuntu, and
> pjsip is probably not packaged because a) it doesn't have that many
> software using it, b) it doesn't build a shared library, as previously
> stated (although there's an intent-to-package python-sipsimple, which
> also embeds pjsip, so there's that problem again).

This is why I started this discussion. I want to know how this is 
fixable. We'd like to do this the "right" way, if at all possible.

As a point of order, I mentioned that resiprocate had limited package 
support. The reason I said this is because, for instance, anyone using 
the current LTS of Ubuntu (12.04) does not have lib-resiprocate-dev 
available to them by default. It was not added until Ubuntu 12.10.

As far as PJSIP's user base is concerned, here's a list of projects from 
the PJSIP site: http://trac.pjsip.org/repos/wiki/Projects_Using_PJSIP . 
That, of course, cannot be considered a complete list. Still, for a SIP 
stack, I wouldn't call that a small list.

The issue is that probably every single one of those projects has had to 
make some modification to PJSIP that would not be beneficial to anyone 
outside of their own project. If PJLIB were to be packaged, then how 
could any of those projects actually use it?
>
>> 2. There is a 100% chance that, no matter which SIP stack we use, we
>> will need to make modifications to suit Asterisk's needs.
> These modifications should go to the upstream SIP stack project. That's
> how the ecosystem works in every other software and library out there --
> that's what Asterisk does with every other library that it uses too.
>
> Can you imagine an ecosystem in which every project embeds and modified
> every other library? Would Asterisk even exist in such an ecosystem?

We would never harbor general-purpose patches for ourselves. All bug 
fixes and general improvements would be pushed upstream so that everyone 
can benefit, no matter whether the library were embedded or not. 
However, not all modifications made would be beneficial to anyone that 
is not using Asterisk. These modifications would have no business being 
pushed upstream.

Expanding the argument to every library is irrelevant here. There are 
many general-purpose libraries that people use and will *never* need to 
modify. SIP stacks, by their very nature, are designed to be modified by 
those that use them.

>> 3. If users are forced to do something arcane in order to be able to
>> upgrade to Asterisk 12 and use the new SIP channel driver, they simply
>> won't do it.
> Depends. I don't see how building a separate library (in case the distro
> is lacking a package) would be so difficult to do for users. Isn't that
> how it goes for libpri, among others?

Libpri is slightly different. If you want PRI support at all, you need 
libpri (or something similar from a third party). You have no 
alternative. Tentatively, the plan for SIP support in Asterisk 12 would 
be to provide both the "classic" chan_sip that Asterisk already includes 
and the new SIP channel driver. Priority would go towards using the new 
channel driver, but the old would be provided as a backup just in case 
you encountered issues.If users get told they can either use no external 
library and have SIP support or install an external library from source 
and have SIP support, they're likely to choose the easier path. This is 
bad since we want as much adoption as possible of the new SIP channel 
driver.

The word "upgrade" was important in my statement above, and I should 
have highlighted it better. If people are using a previous version of 
Asterisk, upgrade, and then get told they need some new library, that 
isn't necessarily a bad thing. If it means installing a distro-included 
package, then it's not a big deal. We did that in Asterisk 10 when we 
required sqlite3 as a new dependency since every single distro includes 
a version of sqlite3. People could simply "yum install" or "apt-get 
install" the new dependency. However, as has already been established, 
distro support for SIP stacks is low to non-existent depending on the 
SIP stack in use. This would mean having to install a new library from 
source to use the new SIP stack.

The typical Asterisk user will not do this. They'll either continue 
using a previous version of Asterisk where things mostly work, or 
they'll upgrade to Asterisk 12 and continue to use the old SIP stack 
instead of the new one.
>
>> 4. Users will not tolerate having to wait for patches to be merged
>> upstream in order to provide fixes for their SIP issues.
> Why? How's it any different than waiting for Asterisk to merge the patches?
>
> The only difference seems to be that this SIP stack wouldn't be in your
> immediate control. If it's a control issue, It sounds to me like you
> don't really want to use a third-party SIP stack.

It's massively different. A user reports a bug in Asterisk today. We 
find the source of the issue. We give them a patch to test. Once the 
patch has been deemed to work and it goes through code review, it gets 
merged into Asterisk. They now have a fixed version of Asterisk locally 
and when the new version comes out, their patch is included in the 
release. Asterisk's developers are the only barrier between a broken and 
fixed version.

A user reports a bug in Asterisk when using a third-party SIP library. 
We find that the issue requires changes to both Asterisk and 
resiprocate/pjsip/sofia. We find the problem and report it to the 
upstream tracker, and we include a patch that fixes the issue. While 
it's certainly possible that they may accept the patch and have it in a 
new version of their library that is available by the time we're ready 
to make a new Asterisk release, it's unlikely. Here are the more likely 
scenarios:

1. The patch submitted by Asterisk to pjsip/resiprocate/sofia needs to 
be refined in some way beyond what was originally uploaded.
2. The patch submitted by Asterisk to pjsip/resiprocate/sofia takes 
longer for their developers to get to than it does before the Asterisk 
team is ready to make a new release of Asterisk.
3. The patch submitted by Asterisk to pjsip/resiprocate/sofia is 
accepted quickly but the next version of their library will not be made 
until well after it is time for Asterisk to make a new release.

(The following two cases assume that the distro package support problem 
has been resolved)

4. The patch submitted by Asterisk to pjsip/resiprocate/sofia is 
accepted quickly and a new release is made by their developers prior to 
Asterisk's next release, but distro packages for that library do not get 
updated until later.
5. The patch submitted by Asterisk to pjsip/resiprocate/sofia is 
accepted quickly and a new release is made by their developers prior to 
Asterisk's next release, and distro packages across all distributions 
are updated as well. However, the user is using a version of his distro 
that does not include the updated version of the package due to policies 
in place by the distro.

In all these cases, the Asterisk developers are not the only barrier to 
the user having a fixed issue.

We absolutely want to use a third-party SIP stack because writing our 
own would be a complete waste of time. 95% of what we would need in a 
SIP stack would be provided by whatever stack we choose to use, and the 
SIP stacks out there are well-tested. There's no point in not building 
off what has already been provided. An apt analogy would be that a 
NASCAR driver doesn't build a car from raw parts. Instead, he takes a 
model that already exists and makes modifications to the engine and the 
body to suit his driving style. In this case, Asterisk is going to need 
to make modifications to suit its "driving style".
>
>> With all of these in mind, what do you suggest the Asterisk project does
>> in order to use a third-party SIP stack?
> Work with the community, both upstreams (SIP stacks) and downstreams
> (distros). Talk with the SIP stack candidates and ask them what their
> feeling is about it. Try to contribute to them so you can affect changes
> on them, get commit access and affect their release schedule.
> Pick a stack based not only its SIP features and compliance, but also
> its integration with the system, the ability to stand as a standard
> library with a well-defined API and ABI, the community behind it and the
> attitude of the maintainers.

Unfortunately, I'm much more pessimistic in endeavours such as these.

Asterisk, the SIP stacks, and distributions all have wildly different 
release cycles. Narrowing it down a bit,  the different distributions 
have wildly different release cycles. And heck, even the different 
distributions that Red Hat provides have different release cycles. If 
we're going to have people rely on distro-provided packages for the SIP 
stack, then that would mean having to compile a table somewhere that 
explains, "If you have issue X, then this is fixed in Asterisk version 
Y, but only if you are either using pjsip/resiprocate/sofia version Z or 
using libpjlib-dev on Ubuntu version A or higher or libpjlib-devel in 
Fedora version B or higher, etc. This is simply unmaintainable and will 
cause mass confusion.

Getting commit access to a SIP stack is easier said than done, and 
trying to affect their release schedule? How likely do you think it is 
that we would be able to affect any of those SIP stacks' release 
schedules? They're being used by dozens of projects, each with their own 
release schedule. Why would Asterisk, a newcomer, have any sway in such 
matters?

All of the SIP stacks have a well-defined API and ABI as it stands. The 
problem is that this only covers 90-95% of peoples' needs in most cases, 
so they add new API calls and change the ABI to suit what they need to 
do. In addition, SIP is rapidly evolving, meaning that new APIs are 
likely to be added to SIP stacks as new versions are released. Most 
libraries function in a domain that has a complete definition and has 
not changed from that initial definition. SIP, on the other hand, is 
constantly undergoing change, and its core RFCs are interpreted 
differently by different people. Trying to define a SIP library that is 
complete, suits everyone's needs, and is easy to use is just not possible.


Mark Michelson

>
> Regards,
> Faidon