[asterisk-bugs] [JIRA] (ASTERISK-18411) Realtime queue members with hints for state_interface get stuck in "In Use" state.

[Sébastien Couture (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-18411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=216016#comment-216016 ] 

Sébastien Couture edited comment on ASTERISK-18411 at 3/3/14 8:13 PM:
----------------------------------------------------------------------

All right, I think I may have figured it out. The issue has to do with dynamic hints with no watchers being lost following a {{dialplan reload}}, and queues hanging on to those hints' last states; it can get stuck in both {{In use}} or {{Not in use}}.

Consider the following dynamic hint, where func_odbc function {{HINT_GET(100)}} returns {{SIP/100}}:
{code}
[agents]
exten => _XXX,hint,${HINT_GET(${EXTEN})}
{code}

Now, if I log agent 100 into queue {{test}} through the CLI (it could be through AddQueueMember, or by adding a SQL entry in the queue_members Realtime table.. makes no difference) and have it use {{hint:100 at agents}} as its {{state_interface}}:
{code}
*CLI> queue add member Local/100 at dial-agent/n to test penalty 0 as 100 state_interface hint:100 at agents
{code}

.. the queue will then internally "follow" the hint's device state and apply it to the agent using app_queue.c's  {{get_queue_member_status()}} function, which gets called when a new agent is added to a queue. This actually triggers the dynamic hints, and creates a specific hint for '100'. Indeed, a {{core show hint 100}} will then show the following:
{code}
*CLI> core show hint 100
100 at agents  :   SIP/100  State:Idle        Watchers  0
{code}

Let's have the agent take a call from the queue:
{code}
*CLI> queue show test
test has 0 calls (max unlimited) in 'ringall' strategy (1s holdtime, 3s talktime), W:0, C:1, A:0, SL:0.0% within 0s
   Members: 
      100 (Local/100 at dial-agent/n from hint:100 at agents) (ringinuse disabled) (dynamic) (In use) has taken 1 calls (last was 11 secs ago)
{code}

Now, if I do a {{dialplan reload}}, hint {{100 at agents}} disappears because Asterisk thinks it's not watched by anything, which isn't totally true because a queue now relies on it to determine one of its agent's state. At that point, the queue just lost what it was using to determine agent 100's state, and when the call ends, the agent's state will be stuck {{In use}}. The fact that the queue has {{ringinuse=no}} here will prevent the agent from getting any more calls.

If I had had a phone subscribe to that hint, and thus had a "watcher", it wouldn't have disappeared and the queue would've still had a device state to use.

I would expect Asterisk to not destroy a hint (or to recreate it) following a {{diaplan reload}} if it has any watchers (which is the case) or if some module "follows" that hint's state internally.

I hope my explanations are somewhat clear.


was (Author: sysreq):
All right, I think I may have figured it out. The issue has to do with dynamic hints with no watchers being lost following a {{dialplan reload}}, and queues hanging on to those hints' last states; it can get stuck in both {{In use}} or {{Not in use}}.

Consider the following dynamic hint, where func_odbc function {{HINT_GET(100)}} returns {{SIP/100}}:
{code}
[agents]
exten => _XXX,hint,${HINT_GET(${EXTEN})}
{code}

Now, if I log agent 100 into queue {{test}} through the CLI (it could be through AddQueueMember, or by adding a SQL entry in the queue_members Realtime table.. makes no difference) and have it use {{hint:100 at agents}} as its {{state_interface}}:
{code}
*CLI> queue add member Local/100 at dial-agent/n to test penalty 0 as 100 state_interface hint:100 at agents
{code}

.. the queue will then internally "follow" the hint's device state and apply it to the agent using app_queue.c's  {{get_queue_member_status()}} function, which gets called when a new agent is added to a queue. This actually triggers the dynamic hints, and creates a specific hint for '100'. Indeed, a {{core show hint 100}} will then show the following:
{code}
*CLI> core show hint 100
100 at agents  :   SIP/100  State:Idle        Watchers  0
{code}

Let's have the agent take a call from the queue:
{code}
*CLI> queue show test
test has 0 calls (max unlimited) in 'ringall' strategy (1s holdtime, 3s talktime), W:0, C:1, A:0, SL:0.0% within 0s
   Members: 
      100 (Local/100 at dial-agent/n from hint:100 at agents) (ringinuse disabled) (dynamic) (In use) has taken 1 calls (last was 11 secs ago)
{code}

Now, if I do a {{dialplan reload}}, hint {{100 at agents}} disappears because Asterisk thinks it's not watched by anything, which isn't totally true because a queue now relies on it to determine one of its agent's state. At that point, the queue just lost what it was using to determine agent 100's state, and when the call ends, the agent's state will be stuck {{In use}}. The fact that the queue has {{ringinuse=no}} here will prevent the agent from getting any other calls.

If I had had a phone subscribe to that hint, and thus had a "watcher", it wouldn't have disappeared and the queue would've still had a device state to use.

I would expect Asterisk to not destroy a hint (or to recreate it) following a {{diaplan reload}} if it has any watchers (which is the case) or if some module "follows" that hint's state internally.

I hope my explanations are somewhat clear.

> Realtime queue members with hints for state_interface get stuck in "In Use" state.
> ----------------------------------------------------------------------------------
>
>                 Key: ASTERISK-18411
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-18411
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Applications/app_queue
>    Affects Versions: 1.8.5.0
>         Environment: 64 bit CentOS
>            Reporter: Steven T. Wheeler
>            Severity: Minor
>
> We have noticed a few times in the last week that the state of queue members has been stuck in "In use" state even though their state interface says "Idle".  When this happens the queue no longer routes calls to the agents.  The only work around we have found so far is to delete the member entries from the realtime table and then add them back in.
> For instance the queue says all the members are In use:
> asterisk -rx 'queue show queue1'
> queue1 has 0 calls (max unlimited) in 'linear' strategy (0s holdtime, 67s talktime), W:0, C:1, A:3, SL:100.0% within 86400s
>    Members:
>       Member 1 (Local/member1 at queue_calling/n) (realtime) (In use) has taken no calls yet
>       Member 2 (Local/member2 at queue_calling/n) (realtime) (In use) has taken 1 calls (last was 6076 secs ago)
>       Member 3 (Local/member3 at queue_calling/n) (realtime) (In use) has taken no calls yet
>       Member 4 (Local/member4 at queue_calling/n) (realtime) (In use) has taken no calls yet
>    No Callers
> But their hints say otherwise:
> asterisk -rx 'core show hints'
>                 member1 at blf                 : SIP/member1_softphon  State:InUse           Watchers  0
>                 member2 at blf                 : SIP/member2_softphon  State:Idle            Watchers  0
>                 member3 at blf                 : SIP/member3_softphon  State:InUse           Watchers  0
>                 member4 at blf                 : SIP/member4_softphon  State:Idle            Watchers  0
> The queue member table configuration:
> mysql> select * from Queue_Members;
> +----------+------------+------------+-------------------------------+------------------+---------+--------+
> | uniqueid | membername | queue_name | interface                     | state_interface  | penalty | paused |
> +----------+------------+------------+-------------------------------+------------------+---------+--------+
> |    31229 | Member 2   | queue1     | Local/member2 at queue_calling/n | hint:member2 at blf |    NULL |   NULL |
> |    31230 | Member 3   | queue1     | Local/member2 at queue_calling/n | hint:member3 at blf |    NULL |   NULL |
> |    31231 | Member 4   | queue1     | Local/member4 at queue_calling/n | hint:member4 at blf |    NULL |   NULL |
> |    31232 | Member 1   | queue1     | Local/member1 at queue_calling/n | hint:member1 at blf |    NULL |   NULL |
> +----------+------------+------------+-------------------------------+------------------+---------+--------+
> This issue occurs infrequently so a debug log would be huge, but if there is more information you need I will try to get it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)