[asterisk-dev] res_fax_spandsp segfaults during fax detection - FIXED?

[Pavel Troller]

Hi!

> Hi folks,
>
> I probably found what causes random segfaults during fax detection with 
> spandsp... Many thanks for your contributions and brainstorming about this, 
> it helped a lot. If you have such issue, please test this possible fix and 
> let me know...
>
> At res_fax.c, somewhere around line 3070, you'll find (at least for 
> Asterisk 11)
>      case AST_FORMAT_ALAW:
>      case AST_FORMAT_ULAW:
> Comment out these two lines, recompile, reload and test.

As I understand the code, by commenting out these lines, we'll just abandon
the frame hook for A/ULAW samples, so we'll never gateway them then... Yes,
it will probabaly stop segfaulting, but as a side-effect, it will stop
gatewaying as well :-). Of course I may be wrong, I just had a little look
there...
  However, it seems that there is really a bug in the hook. 
  - On line 3130, we are attempting to translate the frames before we write
them to the faxing engine (on line 3141). It's OK.
  - BUT: On line 3122, we are calling fax_gateway_detect_v21() with 
UNTRANSLATED frames, which is FATAL - this detection can handle only linear
format similarly as real faxing, so we have to translate non-linear formats
(A/ULAW) for it as well! By not doing this, we probably
1) prevent successful detection of V.21, so it doesn't work either
2) In rare cases, crash the whole thing, because the detection code always
tries to read 320 bytes from 160byte buffer (as Michal pointed out below).
  Once again, I may be wrong, but I think it's in accordance with all the
symptoms we have already collected.

With regards,
  Pavel

>
> These two lines are in framehook code, which is responsible for fax 
> detection for T38gateway. These lines are causing, that ALAW and ULAW audio 
> samples are not transcoded to slinear before they are passed to 
> res_fax_spandsp function spandsp_v21_detect(), and later to libspandsp 
> function modem_connect_tones_rx().  But this libspandsp function doesn't 
> expect ulaw or alaw (8bit) samples, it will read and process samples as 
> they are 16bit slinear.
>
> If we received 160 alaw/ulaw samples (typical 20ms frame) from network, 
> each sample is 1 byte long (sorry for my disinformation about this topic in 
> previous email).  So res_rtp_asterisk allocated 160bytes in the memory for 
> storing this frame (its data part) and gave us a pointer to this memory. If 
> we pass this pointer to function modem_connect_tones_rx() (its second 
> argument) along with information that 160 samples are stored from this 
> position (its third argument), this function is going to read 320 bytes of 
> memory (160 * 16bit) and analyze, if 1100 Hz fax tone is there. But if we 
> allocated and stored 160 bytes in memory, and we are now reading 320bytes, 
> we will get at least bad data or even a segfault too - it depends on 
> position of these data in the memory, which is quite unpredictable, or one 
> may say - random.. Exactly as these segfaults. Elementary my dear 
> Watson.... :o))
>
> --
> Michal Rybarik
>
>