[asterisk-bugs] [JIRA] (ASTERISK-29841) APP Queue Deadlock
Asterisk Team (JIRA)
noreply at issues.asterisk.org
Thu Jan 6 10:13:07 CST 2022
[ https://issues.asterisk.org/jira/browse/ASTERISK-29841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=257658#comment-257658 ]
Asterisk Team commented on ASTERISK-29841:
------------------------------------------
Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.
A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.
Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.
Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].
> APP Queue Deadlock
> ------------------
>
> Key: ASTERISK-29841
> URL: https://issues.asterisk.org/jira/browse/ASTERISK-29841
> Project: Asterisk
> Issue Type: Bug
> Security Level: None
> Components: Applications/app_queue
> Affects Versions: 13.38.3
> Environment: VMWare Virtual Machine
> 16 x vCPU
> 16 GB RAM
> Reporter: André Emerich
> Severity: Major
>
> Hi,
> We are facing a problem with app_queue.
> From time to time Asterisk stops accepting the command "queue show" and it is not possible to give any more commands of this type. As it is a production environment, we enabled DEBUG_THREAD for a short moment and managed to get the evidence below:
> =======================================================================
> === 13.38.3
> === Currently Held Locks
> =======================================================================
> ===
> === <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name> <lock addr> (times locked)
> ===
> === Thread ID: 0x7f8994743700 LWP:23185 (default_tps_processing_function started at [ 228] taskprocessor.c default_listener_start())
> === ---> Lock #0 (app_queue.c): MUTEX 2430 device_state_cb q 0x3206420 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x1c334) [0x7f89adab9334]
> /usr/sbin/asterisk() [0x5d6435]
> /usr/sbin/asterisk() [0x5d6740]
> /usr/sbin/asterisk(ast_taskprocessor_execute+0x167) [0x5f2d27]
> /usr/sbin/asterisk() [0x5f2de0]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === ---> Waiting for Lock #1 (astobj2_container.c): MUTEX 608 internal_ao2_iterator_next iter->c 0x31b6270 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/sbin/asterisk() [0x45dc4d]
> /usr/sbin/asterisk(__ao2_find+0x2d) [0x45e35d]
> /usr/lib64/asterisk/modules/app_queue.so(+0x13a8e) [0x7f89adab0a8e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x162f5) [0x7f89adab32f5]
> /usr/sbin/asterisk(ast_cli_command_full+0x689) [0x4d9209]
> /usr/sbin/asterisk() [0x55e04d]
> /usr/sbin/asterisk() [0x54f5d8]
> /usr/sbin/asterisk() [0x55d15b]
> /usr/sbin/asterisk() [0x5f5eed]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === --- ---> Locked Here: app_queue.c line 9255 (__queues_show)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f894782f700 LWP:24184 (handle_tcptls_connection started at [ 918] tcptls.c ast_tcptls_server_root())
> === ---> Lock #0 (app_queue.c): MUTEX 9255 __queues_show queues 0x31b6270 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x1567b) [0x7f89adab267b]
> /usr/sbin/asterisk(ast_cli_command_full+0x689) [0x4d9209]
> /usr/sbin/asterisk() [0x55e04d]
> /usr/sbin/asterisk() [0x54f5d8]
> /usr/sbin/asterisk() [0x55d15b]
> /usr/sbin/asterisk() [0x5f5eed]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === ---> Waiting for Lock #1 (app_queue.c): MUTEX 9261 __queues_show q 0x3206420 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x9760) [0x7f89adaa6760]
> /usr/lib64/asterisk/modules/app_queue.so(+0x9983) [0x7f89adaa6983]
> /usr/sbin/asterisk(ast_bridge_call_with_flags+0x112) [0x50ffd2]
> /usr/lib64/asterisk/modules/app_queue.so(+0x2aad7) [0x7f89adac7ad7]
> /usr/lib64/asterisk/modules/app_queue.so(+0x2ce38) [0x7f89adac9e38]
> /usr/sbin/asterisk(pbx_exec+0xcc) [0x591e7c]
> /usr/sbin/asterisk() [0x585a8f]
> /usr/sbin/asterisk() [0x587b82]
> /usr/sbin/asterisk() [0x5896db]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === --- ---> Locked Here: app_queue.c line 2430 (device_state_cb)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f89463ad700 LWP:29630 (pbx_thread started at [ 4687] pbx.c ast_pbx_start())
> === ---> Waiting for Lock #0 (app_queue.c): MUTEX 1848 set_queue_variables q 0x3206420 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x9760) [0x7f89adaa6760]
> /usr/lib64/asterisk/modules/app_queue.so(+0x9983) [0x7f89adaa6983]
> /usr/sbin/asterisk(ast_bridge_call_with_flags+0x112) [0x50ffd2]
> /usr/lib64/asterisk/modules/app_queue.so(+0x2aad7) [0x7f89adac7ad7]
> /usr/lib64/asterisk/modules/app_queue.so(+0x2ce38) [0x7f89adac9e38]
> /usr/sbin/asterisk(pbx_exec+0xcc) [0x591e7c]
> /usr/sbin/asterisk() [0x585a8f]
> /usr/sbin/asterisk() [0x587b82]
> /usr/sbin/asterisk() [0x5896db]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === --- ---> Locked Here: app_queue.c line 2430 (device_state_cb)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f8945bdd700 LWP:30356 (pbx_thread started at [ 4687] pbx.c ast_pbx_start())
> === ---> Waiting for Lock #0 (app_queue.c): MUTEX 1848 set_queue_variables q 0x3206420 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x9760) [0x7f89adaa6760]
> /usr/lib64/asterisk/modules/app_queue.so(+0x9983) [0x7f89adaa6983]
> /usr/sbin/asterisk(ast_bridge_call_with_flags+0x112) [0x50ffd2]
> /usr/lib64/asterisk/modules/app_queue.so(+0x2aad7) [0x7f89adac7ad7]
> /usr/lib64/asterisk/modules/app_queue.so(+0x2ce38) [0x7f89adac9e38]
> /usr/sbin/asterisk(pbx_exec+0xcc) [0x591e7c]
> /usr/sbin/asterisk() [0x585a8f]
> /usr/sbin/asterisk() [0x587b82]
> /usr/sbin/asterisk() [0x5896db]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === --- ---> Locked Here: app_queue.c line 2430 (device_state_cb)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f8947fff700 LWP:31161 (pbx_thread started at [ 4687] pbx.c ast_pbx_start())
> === ---> Waiting for Lock #0 (astobj2_container.c): MUTEX 336 internal_ao2_traverse self 0x31b6270 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/sbin/asterisk() [0x45dc4d]
> /usr/sbin/asterisk(__ao2_find+0x2d) [0x45e35d]
> /usr/lib64/asterisk/modules/app_queue.so(+0x13a8e) [0x7f89adab0a8e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x162f5) [0x7f89adab32f5]
> /usr/sbin/asterisk(ast_cli_command_full+0x689) [0x4d9209]
> /usr/sbin/asterisk() [0x55e04d]
> /usr/sbin/asterisk() [0x54f5d8]
> /usr/sbin/asterisk() [0x55d15b]
> /usr/sbin/asterisk() [0x5f5eed]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === --- ---> Locked Here: app_queue.c line 9255 (__queues_show)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f8944bc0700 LWP:2185 (handle_tcptls_connection started at [ 918] tcptls.c ast_tcptls_server_root())
> === ---> Waiting for Lock #0 (astobj2_container.c): MUTEX 608 internal_ao2_iterator_next iter->c 0x31b6270 (1)
> /usr/sbin/asterisk(__ast_pthread_mutex_lock+0x7d) [0x53cf3d]
> /usr/sbin/asterisk(__ao2_lock+0x8e) [0x45bb4e]
> /usr/sbin/asterisk() [0x45dc4d]
> /usr/sbin/asterisk(__ao2_find+0x2d) [0x45e35d]
> /usr/lib64/asterisk/modules/app_queue.so(+0x13a8e) [0x7f89adab0a8e]
> /usr/lib64/asterisk/modules/app_queue.so(+0x162f5) [0x7f89adab32f5]
> /usr/sbin/asterisk(ast_cli_command_full+0x689) [0x4d9209]
> /usr/sbin/asterisk() [0x55e04d]
> /usr/sbin/asterisk() [0x54f5d8]
> /usr/sbin/asterisk() [0x55d15b]
> /usr/sbin/asterisk() [0x5f5eed]
> /usr/sbin/asterisk() [0x605dea]
> /usr/lib64/libpthread.so.0(+0x7ea5) [0x7f8ae852dea5]
> /usr/lib64/libc.so.6(clone+0x6d) [0x7f8ae75cb9fd]
> === --- ---> Locked Here: app_queue.c line 9255 (__queues_show)
> Is there a patch or something to fix this?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
More information about the asterisk-bugs
mailing list