[asterisk-dev] Recommendations for using a SIP stack with Asterisk

[Saúl Ibarra Corretgé]

Hi,

Some replies inline.

Faidon Liambotis wrote:
> On 11/12/12 18:40, Mark Michelson wrote:
>> As a point of order, I mentioned that resiprocate had limited
>> package support. The reason I said this is because, for instance,
>> anyone using the current LTS of Ubuntu (12.04) does not have
>> lib-resiprocate-dev available to them by default. It was not added
>> until Ubuntu 12.10.
>
> As far as I know, you're looking at an Asterisk 12 release at
> tentatively October 2013 and an Asterisk LTS release a year after that.
> Several Ubuntu releases are going to be released until then and the next
> Ubuntu LTS is going to be released in April 2014, not that far away from
> your LTS release (esp. if you drift).
>
>> The issue is that probably every single one of those projects has had
>> to make some modification to PJSIP that would not be beneficial to
>> anyone outside of their own project. If PJLIB were to be packaged,
>> then how could any of those projects actually use it?
>
> Why? How do you know that? How do you know that one of the fixes that
> one of these projects has made, isn't fixing a bug someone will be
> having with Asterisk in two years time and one you're going to end up
> debugging again?
>

I have some experience in this area and I can tell you we had to patch 
PJSIP and there was no way around.

> There's no way to tell! You'd have to manually review all of what is
> effectively active forks of that project in that large list of yours and
> look carefully for fixes relevant to your use cases. Doesn't that suck
> for you? Why would you want to prolong and extend this monstrosity?
>
> Do we really have to argue that merging back changes to the upstream
> project is a good idea?
>
>> However, not all modifications made would be beneficial to anyone
>> that is not using Asterisk. These modifications would have no
>> business being pushed upstream.
>
> Again, how do you know that? You haven't even picked the stack yet, and
> you know how beneficial to others your future patches will be?
>

Of course pushing as much as you can upstream is The Right Way (TM), but 
unfortunately things are not that bright with SIP. You could make a 
perfect SIP library, by the book, and it works in your environment, then 
I try to use it and have to modify it because of interoperability 
problems with broken clients. You can try to push those interop fixes 
upstream, but I were the upstream I'd reject them, they are not correct, 
you need them because the devices you work with are broken, not the library.

And this happens. A hell lot.

>> Expanding the argument to every library is irrelevant here. There
>> are many general-purpose libraries that people use and will *never*
>> need to modify. SIP stacks, by their very nature, are designed to be
>> modified by those that use them.
>
> I don't think it's irrelevant. I don't understand how a SIP stack is
> different from every other library/protocol stack out there. I think
> it's exactly the same.
>

See above.

> This might actually be the case for some of them, and if that's the
> case, I believe they're very badly designed as generic stacks. Do the
> upstreams of all of the three candidate (or possibly other) stacks feel
> like that?
>
>> Libpri is slightly different. If you want PRI support at all, you
>> need libpri (or something similar from a third party). You have no
>> alternative. Tentatively, the plan for SIP support in Asterisk 12
>> would be to provide both the "classic" chan_sip that Asterisk already
>> includes and the new SIP channel driver. Priority would go towards
>> using the new channel driver, but the old would be provided as a
>> backup just in case you encountered issues.If users get told they can
>> either use no external library and have SIP support or install an
>> external library from source and have SIP support, they're likely to
>> choose the easier path. This is bad since we want as much adoption as
>> possible of the new SIP channel driver.
>
> I also don't see how keeping an alternative to the new chan_sip makes
> the situation regarding having a SIP library embedded and forked within
> Asterisk any different. This is completely besides the point.
>
>> However, as has already been established, distro support for SIP
>> stacks is low to non-existent depending on the SIP stack in use. This
>> would mean having to install a new library from source to use the new
>> SIP stack.
>
> As said earlier, I'm fairly certain distributions will catch up long
> before you actually release Asterisk 12, as long as there's a way for
> them to.
>
> BTW, it's nice how you're arguing that SIP stacks are "designed to be
> modified by those that use them" and at the same time complaining about
> distro support. Of course pjsip wouldn't be in distros, that's because
> it's currently impossible! Fix that, and you'll soon find it packaged.
>
>> The typical Asterisk user will not do this. They'll either continue
>> using a previous version of Asterisk where things mostly work, or
>> they'll upgrade to Asterisk 12 and continue to use the old SIP stack
>> instead of the new one.
>
> I think this is a red herring. Not that I'd recommend it, but you could
> always embed an *unmodified*, released version of the library in
> question and never patch it in your tree. Or modify your toolchain to
> fetch it and build it at build time, like you've done in other cases in
> the past.
>

Well, dependencies can be troublesome, look at libstrp. Upstream looks 
abandoned and the version some distros package didn't contain all fixes 
which were on CVS and some of them were pretty critical. Guess what, 
PJSIP bundles it. And PortAudio. And more of them, see third_party/.

>> In all these cases, the Asterisk developers are not the only barrier
>> to the user having a fixed issue.
>
> I think that's the core of the issue, as I mentioned in my previous
> mail. You want to have control; working with the community means giving
> up control, so you're essentially in an impasse that you want to resolve
> by forking and embedding.
>

I think both can still be achieved. Asterisk could use a bundles and 
patched version while contributing uostream and eventually getting rid 
of the patches that are accepted. We may even see a day were PJSIP 
accepts all patches and Asterisk doesn't need to bundle a custom 
version! But having that as a starting point will be a no-go IMHO.

>> We absolutely want to use a third-party SIP stack because writing
>> our own would be a complete waste of time. 95% of what we would need
>> in a SIP stack would be provided by whatever stack we choose to use,
>> and the SIP stacks out there are well-tested.
>
> This is a bit besides the point but: if they're well-tested and they've
> proven to be better than you at this, then why do you argue that you
> should be patching their code, bypassing upstream's review&  testing
> process? :-)
>

Because SIP fucked up sometimes, and you need two broken implementations 
so that they work together :-)

> You're effectively admitting failure in building a SIP stack and you're
> throwing away years of work on chan_sip only to replace it with
> something someone else built and that you (rightfully) want to re-use.
>
> At the same time, you're arguing that you already "know better" from the
> upstream (the one that you haven't even picked up yet!), and you'd
> prefer to patch their code unhindered. How's that going to be better
> than the current chan_sip in the long run?
>

I can't speak for the Asterisk project, but in my experience, yes, 
patching PJSIP was needed. We contributed everything upstream, some 
patches are still there for 3-4 years. And some others are not 
applicable upstream because we modified APIs to our needs and broke 
other APIs we don't use / need.

>> Unfortunately, I'm much more pessimistic in endeavours such as
>> these.
>>
>> Asterisk, the SIP stacks, and distributions all have wildly
>> different release cycles. Narrowing it down a bit,  the different
>> distributions have wildly different release cycles. And heck, even
>> the different distributions that Red Hat provides have different
>> release cycles. If we're going to have people rely on distro-provided
>> packages for the SIP stack, then that would mean having to compile a
>> table somewhere that explains, "If you have issue X, then this is
>> fixed in Asterisk version Y, but only if you are either using
>> pjsip/resiprocate/sofia version Z or using libpjlib-dev on Ubuntu
>> version A or higher or libpjlib-devel in Fedora version B or higher,
>> etc. This is simply unmaintainable and will cause mass confusion.
>
> You're confusing two different matters for the third time in your mail:
> forking/embedding vs. pushing your changes to upstream is completely
> orthogonal to what distributions, their packages and release schedules do.
>
>> All of the SIP stacks have a well-defined API and ABI as it stands.
>> The problem is that this only covers 90-95% of peoples' needs in most
>> cases, so they add new API calls and change the ABI to suit what they
>> need to do.
>
> That's an oxymoron. This 5-10% of people's needs can be pushed back into
> upstream's well-defined API/ABI. If everyone's doing it differently,
> then it's not a well-defined API/ABI anymore.
>
> Or, let me put it this way: let's say you pick pjsip; you fork it, add
> the 10% API calls that you're missing and other bugfixes of yours. What
> stops you from releasing that as a separate tarball, that builds a
> separate shared library called, say, libfoosip? Would it have a
> well-defined API/ABI?
>
>> In addition, SIP is rapidly evolving, meaning that new
>> APIs are likely to be added to SIP stacks as new versions are
>> released. Most libraries function in a domain that has a complete
>> definition and has not changed from that initial definition. SIP, on
>> the other hand, is constantly undergoing change, and its core RFCs
>> are interpreted differently by different people. Trying to define a
>> SIP library that is complete, suits everyone's needs, and is easy to
>> use is just not possible.
>
> I don't see how these changes can't be implemented in a shared library
> as new API calls for everyone to use and profit from. More features in
> SIP stacks, there for the next project to use, test and contribute back to!
>
> And I think you're vastly overestimating SIP; SIP is indeed a complex
> protocool but it's not more so than *everything else out there*. Just
> look at the thousands of libraries present in your average distribution
> and the protocols that they implement.
>

I think you are over optimistic about how PJSIP's internals work and how 
easy it is to integrate it with another project. While I agree that what 
you propose is a great thing I think that the more pragmatic approach 
proposed by Mark is more realistic.


Regards,

-- 
Saúl Ibarra Corretgé
http://saghul.net/blog | http://about.me/saghul