[asterisk-dev] Segfault in ast_readaudio_callback - Possible cause and solution.

[Steve Davies]

Hi,

Please don't discard this email out of hand when I say I have only
reproduced it in 1.6.2.22 - I believe this is a flaw in 1.8 and 10
that will be of interest! This crash has been mentioned in a few old
reports, for example #ASTERISK-16975.

Cause:

- SIP Refer (attended transfer) a caller into a Queue while the
position announce is playing.
- The crash happens more easily on a fast multi-core server.

Under-the-hood:

The Queue position announce is being played using
ast_readaudio_callback, which is called with an ast_filestream*,
usually every 20ms.

In the meantime, The SIP Refer sets up a masquerade from within a
separate thread. If that masquerade is picked-up by ast_read() or
ast_waitfor_nandfs(), then all is happy, BUT, the
ast_readaudio_callback() function uses ast_write, which can also
trigger the ast_do_masquerade().

Depending on thread scheduling, the call to ast_do_masquerade can
cause the stream's channel to be freed, or sometimes just marked as a
ZOMBIE, but either way, the ast_filestream is usually closed using
ast_closestream(), setting it's ao2 refcount to zero, destroying it,
and allowing the memory to be re-used.

At this point execution passes back from the ast_write() to
ast_readaudio_callback() where it tries to use the freed
ast_filestream, which in my testing, generally points at garbage data,
and if it survives that experience, it tries to access the freed
channel instead.

Proposed solution:

1) Increase the refcount on the ast_filestream while in
ast_readaudio_callback so that it is not freed from under it.
2) When calling ast_closestream(), set the stream->owner to NULL so it
is clear that the channel should no longer be used by that stream.

My greatest concern is that I have no feeling for how much extra load
ao2_ref/ao2_unref adds? I've attached a sample patch (for 1.6.2...
sorry!), which I can upload and disclaim if anyone thinks that it is
useful to do so. I have a site where this issue happens once a day at
the moment!

Thoughts please?

Regards,
Steve

PS. Yes, I am trying to get us moved up to Asterisk 1.8 ASAP :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: readaudio_callback_workaround
Type: application/octet-stream
Size: 2370 bytes
Desc: not available
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20120315/e991093c/attachment.obj>