[asterisk-dev] State of FAX (primarily T.38) in Asterisk trunk (planning for 1.8 release)

[Kevin P. Fleming]

Over the past year, some of us at Digium have spent many (MANY) hours
working on FAX support in Asterisk (even though we'd all prefer to see
FAX go away as the obsolete technology that it should be <G>). Some of
this work was part of our release of our commercial FAX for Asterisk
product, some of it was driven by our desires to have solid FAX support
in our commercial PBX products, but most of it was driven by the need to
get open source Asterisk (and its community of users) able to use FAX
reliably with as many non-Asterisk endpoints as possible.

In Asterisk 1.4 and prior releases, there was limited support for FAX of
any kind; Asterisk 1.4 can pass through T.38 sessions on SIP channels,
but that's all. With an open source addon (based on Steve Underwood's
excellent spandsp library), it can have dialplan applications to send
and receive FAX over audio (G.711 TDM) links.

During the development of Asterisk 1.6.0, Steve's FAX dialplan
applications were merged into asterisk-addons, and then directly into
the Asterisk source tree, where they became the 'app_fax' module. In
addition, the T.38 negotiation process was redesigned to allow Asterisk
applications to actually be T.38 endpoints; this resulted in the ability
to send and receive FAX over audio *and* T.38 links.

Once this got into the community's hands, we started seeing large
numbers of bug reports because users could not successfully FAX over
T.38 with various ATAs and service providers. I won't go into the gory
details of why this was the case, except to repeat a recent quote from
Steve himself on IRC: "The T.38 spec was written after a night of heavy
drinking as a joke.". While that's not technically true, it is true that
compliance with the T.38 recommendation, primarily in the SIP/SDP
negotiations area, is very poor across the industry. Producing a T.38
endpoint the interoperates widely is a complex and difficult process,
that can only be achieved through hours and hours (and hours) of testing.

As these problem reports got more severe, we took a significant step: we
rewrote the T.38 negotiation mechanisms in Asterisk *again*. These
changes first appeared in the 1.6.0.14 and 1.6.1.5 releases. The in-tree
app_fax application was updated to support these changes along with
them, so open source FAX users got the benefits of these changes
immediately... and the result has been wonderful. We get very few T.38
negotiation-related bug reports now, and in nearly every case we can
point to a misconfiguration or severely broken far endpoint that is the
cause of the problem. For many, many people, FAX over T.38 in Asterisk
1.6.0 and 1.6.1 'just works' now.

While all of this was going on, Digium was also working on our
commercial Fax for Asterisk product, which provides comparable
functionality to app_fax, but uses a commercial FAX stack. When we began
the development of this product, we knew that we wanted as many portions
of it as possible to end up open source, so rather than build it as a
monolithic module, we built two modules: res_fax and res_fax_digium.
res_fax is similar to app_fax, in that it provides dialplan
applications, dialplan functions, and other associated components to
send and receive FAX. However, it does not actually include any FAX
stack; for that, it provides a plugin interface that allows one (or
more) additional modules to be loaded to actually provide the FAX
technology implementation. It *does* handle all T.38 negotiations,
however, using the 3rd generation T.38 implementation in Asterisk.

Of course, res_fax_digium is Digium's commercial plugin module for
res_fax, and we have been delivering the combination of those two
modules to our Fax For Asterisk customers for quite a few months now.
(Unfortunately it has taken longer to get these modules updated to match
Asterisk's T.38 API than we would have liked, but that's not important
for open source users). Recently, with a final set of changes to the
UDPTL stack in Asterisk, we've reached the point where we think we've
implemented all the parts of the T.38 recommendation that we can
implement, in as compatible and interoperable a way as we can, with a
few configuration options that provide the ability to override broken
behavior by far endpoints when necessary. In fact, the combination of
the very most recent Asterisk open source releases and either app_fax or
res_fax+res_fax_digium now successfully interoperates with quite a few
T.38 ATAs we have in our lab and a couple of service providers, and
we're testing more of both as I write this. In nearly every case, this
works *without* requiring any configuration changes in Asterisk or the
FAX applications, which is good news for end users indeed. We've even
extended (and fixed) the 'faxdetect' functionality in chan_sip so that
it works as users expect it to, in a similar way to the faxdetect
feature in chan_dahdi.

Now that this point has been reached, it's time for us to act on our
plan to open source res_fax; to that end, I've created a new branch in
Subversion (/svn/asterisk/team/group/res_fax) based on Asterisk trunk,
and merged the most recent version of res_fax into it, updating it to
compile against trunk's recent API changes. By the time Asterisk 1.8 is
released, res_fax will *replace* app_fax, as it provides the identical
dialplan applications (same names, options and operations), but has
additional features and compatibility functionality. To achieve this,
though, we're going to need a res_fax_spandsp plugin module, so that the
combination of res_fax + res_fax_spandsp provides the same functionality
as app_fax did, and it will be a seamless transition for Asterisk 1.6.x
FAX users to move to Asterisk 1.8 when they are ready. We've tasked one
of our developers to start working on this over the next few weeks, and
I'm sure you'll see some initial steps toward that end shortly. For
those of you who have contributed improvements to app_fax and use it
heavily (or even those who just use it and can provide testing), I'd
encourage you to reply in this thread if you are interested and willing
to assist in this effort.

Now, on to even more interesting stuff: as I've worked on this over the
past 6-8 months, I've also spent time finally trying to understand how
best to fit TDM-T.38 FAX relay into Asterisk. I know that there are
number of community developers who have been working on this, and there
are multiple patches in the issue tracker that provide this
functionality in various forms. There was also an attempt by Daniel
Ferenci to start a discussion on this list about the best long-term
approach for relay support, but he didn't really get any responses.

Once res_fax and res_fax_spandsp are ready for users to use, it'll be
time for them to be extended to support FAX relay as well. I would like
to propose that this be implemented by res_fax providing a new API, a
'FAX relay session', that a channel driver (one which services TDM
channels) can use to offer T.38 support to the rest of Asterisk, as if
it supported T.38 natively. We can start a separate thread here to talk
about particulars, but I believe this is the cleanest approach with the
least impact on existing code in Asterisk, and I'd like to get opinions
from other interested parties and discussion going.

In summary, it appears that Asterisk 1.6.x has achieved pretty solid FAX
support (especially over T.38), and that with Asterisk 1.8 we'll be able
to improve it further and begin moving towards FAX relay support as
well. For all of you who still insist on using this obsolete technology,
I hope this has provided what you need to be able to keep using Asterisk
in every place that it makes sense in your networks!

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
skype: kpfleming | jabber: kpfleming at digium.com
Check us out at www.digium.com & www.asterisk.org