[Asterisk-bugs] [Asterisk 0010199]: Random replacement of channel name with other text in queue log entries

[noreply at bugs.digium.com]

A NOTE has been added to this issue. 
====================================================================== 
http://bugs.digium.com/view.php?id=10199 
====================================================================== 
Reported By:                jfitzgibbon
Assigned To:                
====================================================================== 
Project:                    Asterisk
Issue ID:                   10199
Category:                   Applications/app_queue
Reproducibility:            random
Severity:                   major
Priority:                   normal
Status:                     feedback
Asterisk Version:           1.4.7.1  
SVN Branch (only for SVN checkouts, not tarball releases): N/A  
SVN Revision (number only!):  
Disclaimer on File?:        No 
Request Review:              
====================================================================== 
Date Submitted:             07-13-2007 08:44 CDT
Last Modified:              07-18-2007 17:03 CDT
====================================================================== 
Summary:                    Random replacement of channel name with other text
in queue log entries
Description: 
On 1.4.5, 1.4.6 and 1.4.7.1, I have observed random replacement of the
channel name with other text in my queue log.  Typically the other text is
part of a manager event, suggesting that a pointer is getting corrupted
somewhere.  It happens on a very small percentage of calls, but there are
consistent elements to the corruption from my observations:

- it only happens on one of my eleven queues.  Of the eleven, four others
are configured almost identically to the queue on which the corruption is
observed (the only differences are queue name and wrapuptime length). The
problem queue is listed second in queues.conf, and does not sort lexically
to the top or bottom of a list of queue names.

- the corruption first appears on the CONNECT event

- the channel name remains corrupted for the duration of the call (i.e.
you never see a good ENTERQUEUE, bad CONNECT, then a good COMPLETECALLER)

- the replacement text is not consistent.  It can differ between the
CONNECT and TRANSFER events (see example 1)

- if the call is transferred to an extension which enqueues the caller to
another queue, the corruption is cleared, and does not typically re-appear
(see example 1)

I cannot reproduce the bug on demand in my lab environment.  On a call
center fielding about 5000 calls per day, I see this on 5-10 of those
calls.  The main impact of this is that queue analysis programs never see
the CONNECT or COMPLETEXXXXXX events, so they think that any corrupted call
is waiting forever.  Having such a large wait time for several calls on a
queue knocks statistics out of whack.

====================================================================== 

---------------------------------------------------------------------- 
 stuarth - 07-18-07 17:03  
---------------------------------------------------------------------- 
Seems it happens a lot more often to you than me, then... Since the first
version you mentioned was 1.4.5 I went looking for changes between 1.4.4
and there; I did wonder about a free() to fix a memory leak (r65394):

Fix a memory leak that I just noticed in the device state
handling in app_queue. On most device state changes, it would
leak roughly 8 to 64 bytes (the length of the name of the
device). ........

I didn't trace it far enough to tell if there was a use-after-free
problem, but the fact that it's the device name, and was introduced between
1.4.4 and 1.4.5 did make me wonder. You might want to try reverting it,
then you can at least blow a hole in my theory (-:

--- apps/app_queue.c.orig    2007-07-18 23:12:24.000000000 +0100
+++ apps/app_queue.c 2007-07-18 00:44:23.000000000 +0100
@@ -584,13 +584,13 @@ static void *changethread(void *data)
                        }
                }
                ast_mutex_unlock(&q->lock);
        }
        AST_LIST_UNLOCK(&queues);
 
-       free(sc);
+       /* XXX ??? free(sc); */
 
        return NULL;
 }
 
 static int statechange_queue(const char *dev, int state, void *ign)
 { 

Issue History 
Date Modified   Username       Field                    Change               
====================================================================== 
07-18-07 17:03  stuarth        Note Added: 0067558                          
======================================================================