[asterisk-dev] res_fax_spandsp segfaults during fax detection

[Michal Rybárik]

Hello,

I have problem with random Asterisk segfaults on the machine, which I 
use as T.38 gateway between DAHDI and SIP. I would like to kindly ask 
somebody to take a look at it, and help me to find what's wrong... 
Asterisk is version 11 from SVN, r382022 (I'm using this because of 
other dependencies - I compared relevant sources to current v11 SVN and 
they are almost unchanged).

Segfault happens on voice calls, during detection of fax preamble. 
Segfaults happens randomly - sometimes there is segfault after 50.000 
calls, sometimes after 5 calls. In coredumps I see, that segfault 
happens in libspandsp2.so (version 0.06-pre21, and latest snapshot too).

I asked Steve Underwood (spandsp author) about this, and he pointed me 
to the application itself - probably there is something wrong with "amp" 
(pointer to the audio samples data), because this pointer is first time 
used in function fsk_rx(), where segfault happens. So I looked deeper 
into this, and added some debug info into the res_fax_spandsp.c source, 
into function spandsp_v21_detect(), just before calling 
modem_connect_tones_rx() (the function, which calls fsk_rx() later). Now 
I see the contents of frame which caused segfault, and also the "amp" 
pointer (in asterisk it is f->data.ptr), but I'm not sure what's wrong 
with it.

[Jan 27 14:00:22] VERBOSE[30694][C-000006cb] app_dial.c:     -- Called 
DAHDI/G2/123456789
[Jan 27 14:00:27] VERBOSE[30694][C-000006cb] app_dial.c:     -- 
DAHDI/57-1 is proceeding passing it to SIP/mypbx-00000729
[Jan 27 14:00:27] VERBOSE[30694][C-000006cb] app_dial.c:     -- 
DAHDI/57-1 is ringing
[Jan 27 14:00:32] VERBOSE[30694][C-000006cb] app_dial.c:     -- 
DAHDI/57-1 answered SIP/mypbx-00000729
[Jan 27 14:00:32] NOTICE[30694][C-000006cb] res_fax_spandsp.c: frame={ 
frametype=2, datalen=320, samples=160, mallocd=1, mallocd_hdr_len=722, 
offset=64, src=alawtolin, flags=0, ts=0, len=0, seqno=0, 
data.ptr=0xb50c91b8  }
[Jan 27 14:00:32] NOTICE[30694][C-000006cb] res_fax_spandsp.c: frame={ 
frametype=2, datalen=160, samples=160, mallocd=1, mallocd_hdr_len=562, 
offset=64, src=RTP, flags=1, ts=9140, len=20, seqno=1489, 
data.ptr=0xb4ef4f30  }
  (... segfault now ...)

Core was generated by `/usr/sbin/asterisk -f -p -U asterisk -vvvg -c'.
Program terminated with signal 11, Segmentation fault.
#0  fsk_rx (s=0x83ea7e8, amp=0xb4ef4f30, len=160) at fsk.c:381
381                 s->window[j][buf_ptr].re = (ph.re*amp[i]) >> 
s->scaling_shift;

Last line from Asterisk log shows contents of ast_frame struct *f, which 
caused segfault. I see that segfualt was caused by first frame, which 
arrived from DAHDI (src=RTP) and which was passed to 
spands_v21_detect(), and then to modem_connect_tones_rx(), and then 
fsk_rx().

Only one unusual thing, which I see on this frame, is that 
f->mallocd_hdr_len=562. Many other frames have this set to 722 (if 
f->mallocd==1) or to 0 (if f->mallocd==0). But in a few cases, I saw 
frames with malloc_hdr_len set to different values, and these frames 
didn't cause segfault.

Is there anybody who can help?
Many thanks..

--
Michal Rybarik