[asterisk-bugs] [JIRA] (ASTERISK-29578) app_queue: Custom device states or hints become stale and fail to update
Joshua C. Colp (JIRA)
noreply at issues.asterisk.org
Sun Sep 5 12:34:33 CDT 2021
[ https://issues.asterisk.org/jira/browse/ASTERISK-29578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=256159#comment-256159 ]
Joshua C. Colp commented on ASTERISK-29578:
-------------------------------------------
I'm not actively pursuing this issue or digging into it. A hint is treated differently in the code, and uses extension state instead[1]. The app_queue module does NOT directly receive updates for anything (devices) that the hint may comprise of. It receives extension state updates[2].
[1] https://github.com/asterisk/asterisk/blob/master/apps/app_queue.c#L2718
[2] https://github.com/asterisk/asterisk/blob/master/apps/app_queue.c#L2633
> app_queue: Custom device states or hints become stale and fail to update
> ------------------------------------------------------------------------
>
> Key: ASTERISK-29578
> URL: https://issues.asterisk.org/jira/browse/ASTERISK-29578
> Project: Asterisk
> Issue Type: Bug
> Security Level: None
> Components: Applications/app_queue
> Affects Versions: 18.4.0
> Environment: Debian 10
> Tested on Asterisk 18.4 with chan_pjsip and Asterisk 18.3 with chan_sip, stock queues.conf configuration. Bug appears in both.
> Reporter: N A
> Assignee: Unassigned
> Attachments: cli.txt, debug_log_29578.txt, queues.conf
>
>
> I have been encountering an issue for several weeks now where the device state that app_queue has is consistently outdated.
> I am using statically configured queue members in queues.conf - the queue members themselves are not realtime, dynamic, etc.
> Things will work fine initially, but eventually - and especially if a device was temporarily unavailable and is back online later - the device state that app_queue has will become outdated and inaccurate. Here is a recent call, for instance, where I manually use ${DEVICE_STATE()} to check the state of a member in the queue:
> [2021-08-12 23:03:00] -- Executing [s at queue:2] NoOp("IAX2/8142", "NOT_INUSE") in new stack
> [2021-08-12 23:03:00] -- Executing [s at queue:3] Set("IAX2/8142", "__queuechannel=IAX2/8142") in new stack
> [2021-08-12 23:03:00] -- Executing [s at queue:4] Set("IAX2/8142", "QUEUENAME=test") in new stack
> [2021-08-12 23:03:00] WARNING[6505][C-000002bc]: app_queue.c:8356 queue_exec: Unable to join queue 'test'
> If I run "queue show", all the queues with that member show "Unavailable".
> Running "queue reload all" alone will not fix the issue, because app_queue is smart enough to realize the config file has not changed.
> Running "touch queues.conf" and *then* "queue reload members" will fix the issue, forcing app_queue to purge its stale device status and then things are finally updated... for the moment. Before long, everything is stale again.
> The problem seems to be that app_queue is consistently wrong and stale to the point where the application is unusable, and unless I call System(touch queues.conf; asterisk -rx "queue reload members") before any caller enters a queue, there is currently no way around this.
> This all was using PJSIP + Asterisk 18.4. To try to replicate this, I also tested on an Asterisk 18.3 system using SIP.
> If I add devices directly as queue members, this issue does not seem to come up. However, if I use hints for device state, things start to become less reliable there:
> [Aug 12 20:09:40] -- Executing [223 at local:1] NoOp("IAX2/14159", "NOT_INUSE") in new stack
> PBX*CLI> core show hints
> -= Registered Asterisk Dial Plan Hints =-
> 231 at test-hints : SIP/ATAxOffice1 State:Unavailable Presence:not_set Watchers 1
> So, again DEVICE_STATE = NOT_INUSE, but hint state is Unavailable.
> This is the hint: exten => 2127,hint,PJSIP/ATAxLA2
> So perhaps this isn't an issue with queues, really, but an issue with the hints that app_queue is using not aligning with the actual device state. That's what it's starting to look like to me.
> 2127 at hints-cen: PJSIP/ATAxLA2 State:Idle Presence:not_set Watchers 6
> test has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 12s
> Members:
> PJSIP/ATAxLA2 (ringinuse disabled) (Not in use) has taken no calls yet
> 231 (Local/115 at phreaknet-queue-ring from hint:2127 at hints-all) with penalty 5 (ringinuse disabled) (Unavailable) has taken no calls yet
> No Callers
> Okay, so now let's add a member to the queue and do "queue reload members":
> test has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 12s
> Members:
> PJSIP/ATAxLA2 (ringinuse disabled) (Not in use) has taken no calls yet
> 231 (Local/116 at queue-ring from PJSIP/ATAxLA2) with penalty 5 (ringinuse disabled) (Not in use) has taken no calls yet
> 231 (Local/115 at queue-ring from hint:2127 at hints-all) with penalty 5 (ringinuse disabled) (Not in use) has taken no calls yet
> No Callers
> Aha! So you see, when app_queue reloads members, the stale hints are refreshed. In this case, I've added a local channel but with device state to a device rather than a hint, so we can now compare all 3 cases. Many hours later:
> 2127 at hints-cen: PJSIP/ATAxLA2 State:Idle Presence:not_set Watchers 6
> test has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 12s
> Members:
> PJSIP/ATAxLA2 (ringinuse disabled) (Not in use) has taken no calls yet
> 231 (Local/116 at queue-ring from PJSIP/ATAxLA2) with penalty 5 (ringinuse disabled) (Unavailable) has taken no calls yet
> 231 (Local/115 at queue-ring from hint:2127 at hints-all) with penalty 5 (ringinuse disabled) (Unavailable) has taken no calls yet
> No Callers
> The device is currently available. I have added the same endpoint 3 different ways. app_queue things one of them is available (specified with PJSIP as the direct tech), and the other two, which are local channels using a custom hint or device state, are unavailable.
> So, it doesn't matter whether a device is specified directly or a hint extension is used. Both suffer from the same problem. All three queue members above correspond to literally the same endpoint. The end device state should be equivalent, but in fact, the device state is only accurate if a real channel type is used for the queue member. If a hint or device is specified, it becomes stale and inaccurate.
> So as can be seen, this doesn't affect all types of queue members and it isn't a bug with hints themselves. Rather, it seems to be an issue specifically with the way that app_queue figures out device state from hints. Initially, it's right, but over time, it becomes stale and basically screws up everything if that is how you are determining if queue members are available or not.
> I tried this on another server using Asterisk 18.3 and chan_sip with a stock queues.conf config (except for the members):
> [test-queue]
> member => SIP/ATAxOffice1
> member => Local/419 at public,0,Conference,SIP/ATAxOffice1
> member => Local/418 at public,0,Conference,hint:231 at test-hints
> I do notice some of the same issues. For instance, this after a peer becomes unavailable:
> PBX*CLI> queue show
> msr has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> No Members
> No Callers
> test-queue has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> Members:
> SIP/ATAxOffice1 (ringinuse enabled) (Not in use) has taken no calls yet
> Conference (Local/419 at public from SIP/ATAxOffice1) (ringinuse enabled) (Not in use) has taken no calls yet
> Conference (Local/418 at public from hint:231 at test-hints) (ringinuse enabled) (Not in use) has taken no calls yet
> No Callers
> [Aug 15 12:11:39] NOTICE[805]: chan_sip.c:30516 sip_poke_noanswer: Peer 'ATAxOffice1' is now UNREACHABLE! Last qualify: 12
> [Aug 15 12:11:39] NOTICE[805]: chan_sip.c:30516 sip_poke_noanswer: Peer 'ATAxOffice2' is now UNREACHABLE! Last qualify: 10
> PBX*CLI> queue show
> msr has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> No Members
> No Callers
> test-queue has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> Members:
> SIP/ATAxOffice1 (ringinuse enabled) (Unavailable) has taken no calls yet
> Conference (Local/419 at public from SIP/ATAxOffice1) (ringinuse enabled) (Not in use) has taken no calls yet
> Conference (Local/418 at public from hint:231 at test-hints) (ringinuse enabled) (Unavailable) has taken no calls yet
> No Callers
> PBX*CLI> queue show
> msr has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> No Members
> No Callers
> test-queue has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> Members:
> SIP/ATAxOffice1 (ringinuse enabled) (Unavailable) has taken no calls yet
> Conference (Local/419 at public from SIP/ATAxOffice1) (ringinuse enabled) (Not in use) has taken no calls yet
> Conference (Local/418 at public from hint:231 at test-hints) (ringinuse enabled) (Unavailable) has taken no calls yet
> No Callers
> All three of these agents have the same device state, but right after one becomes unavailable, 2 of them correctly show "Unavailable" while one of them is still "Not in use" (which is, actually, the opposite problem of the above). If I touch the file, then do "queue reload all", the middle one then correctly changes to "Unavailable". But watch what happens once the peers come back online:
> test-queue has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> Members:
> SIP/ATAxOffice1 (ringinuse enabled) (Unavailable) has taken no calls yet
> Conference (Local/419 at public from SIP/ATAxOffice1) (ringinuse enabled) (Unavailable) has taken no calls yet
> Conference (Local/418 at public from hint:231 at test-hints) (ringinuse enabled) (Unavailable) has taken no calls yet
> No Callers
> [Aug 15 12:17:40] NOTICE[805]: chan_sip.c:24984 handle_response_peerpoke: Peer 'ATAxOffice1' is now Reachable. (12ms / 2000ms)
> [Aug 15 12:17:40] NOTICE[805]: chan_sip.c:24984 handle_response_peerpoke: Peer 'ATAxOffice2' is now Reachable. (11ms / 2000ms)
> PBX*CLI> queue show
> msr has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> No Members
> No Callers
> test-queue has 0 calls (max unlimited) in 'ringall' strategy (0s holdtime, 0s talktime), W:0, C:0, A:0, SL:0.0%, SL2:0.0% within 0s
> Members:
> SIP/ATAxOffice1 (ringinuse enabled) (Not in use) has taken no calls yet
> Conference (Local/419 at public from SIP/ATAxOffice1) (ringinuse enabled) (Unavailable) has taken no calls yet
> Conference (Local/418 at public from hint:231 at test-hints) (ringinuse enabled) (Not in use) has taken no calls yet
> No Callers
> So on this particular Asterisk, it seems that the middle agent is consistently getting "stuck" in a stale state. Channel driver does not seem to matter - all that it takes is specifying custom device states. Hence, it seems there are general problems with app_queue itself with using custom device states/hints for availability.
> This kind of thing now happened dozens of times in the past month. Since I need to use local channels for my queue members, this effectively makes app_queue completely unusable for me.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
More information about the asterisk-bugs
mailing list