[asterisk-bugs] [JIRA] (ASTERISK-30157) Deadlock between bridge_channel_internal_join and ao2 channel traversal

Joshua C. Colp (JIRA) noreply at issues.asterisk.org
Tue Jul 26 09:03:08 CDT 2022


    [ https://issues.asterisk.org/jira/browse/ASTERISK-30157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=259804#comment-259804 ] 

Joshua C. Colp commented on ASTERISK-30157:
-------------------------------------------

I'd want to see a backtrace without IAX2 involved. One of the threads deadlocked and holding a lock is as a result of IAX2 device state, when going through the channels since it doesn't natively support it.

> Deadlock between bridge_channel_internal_join and ao2 channel traversal
> -----------------------------------------------------------------------
>
>                 Key: ASTERISK-30157
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-30157
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Channels/chan_iax2, Core/Channels
>    Affects Versions: 18.9.0
>            Reporter: N A
>         Attachments: 6bzz1npcct.txt
>
>
> This has happened several times now that I've begun to pick out a pattern of this deadlock consistently happening in the same place, so here we are:
> The deadlock is always on a channel (which then blocks the entire channels container), which basically grinds Asterisk to a halt as existing calls may complete but no new channels can be created.
> core show locks:
> {noformat}
> [2022-07-26 10:45:51] ERROR[7806]: lock.c:76 __dump_backtrace: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f20923beeff]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:299 __ast_pthread_mutex_lock: channel.c line 922 (__ast_channel_alloc_ap): Deadlock? waited 16665 sec for mutex 'channels'?
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ast_bt_get_addresses+0x1d) [0x5603f1ac6716]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb5) [0x5603f1b81517]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ao2_lock+0x10b) [0x5603f1ab9091]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0xb6c6e) [0x5603f1af8c6e]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ast_channel_alloc+0xde) [0x5603f1af8f56]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_unreal_new_channels+0x227) [0x5603f1b3d14c]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0xf7195) [0x5603f1b39195]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0xc8cfa) [0x5603f1b0acfa]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_request+0x51) [0x5603f1b0b3ca]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x103352) [0x5603f1b45352]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_dial_prerun+0x77) [0x5603f1b45845]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x17395e) [0x5603f1bb595e]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_pbx_outgoing_app_predial+0xb9) [0x5603f1bb64a6]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_pbx_outgoing_app+0x5d) [0x5603f1bb63e7]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/lib/asterisk/modules/pbx_spool.so(+0x4f24) [0x7f2083a3df24]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x215820) [0x5603f1c57820]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3) [0x7f209292afa3]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f20923beeff]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:307 __ast_pthread_mutex_lock: astobj2_container.c line 310 (internal_ao2_traverse): 'channels' was locked here.
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ast_bt_get_addresses+0x1d) [0x5603f1ac6716]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ast_pthread_mutex_lock+0xb5) [0x5603f1b81517]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ao2_lock+0x10b) [0x5603f1ab9091]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x79982) [0x5603f1abb982]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(__ao2_callback_data+0x4d) [0x5603f1abbd37]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_channel_callback+0x54) [0x5603f1afa0c8]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_channel_get_by_name_prefix+0x46) [0x5603f1afa7f9]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(ast_parse_device_state+0x4e) [0x5603f1b42caa]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x10102b) [0x5603f1b4302b]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x1014d1) [0x5603f1b434d1]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x1018aa) [0x5603f1b438aa]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /usr/sbin/asterisk(+0x215820) [0x5603f1c57820]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3) [0x7f209292afa3]
> [2022-07-26 10:45:51] ERROR[10667]: lock.c:76 __dump_backtrace: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f20923beeff]
> voip*CLI> core show locks
> =======================================================================
> === 18.9.0
> === Currently Held Locks
> =======================================================================
> ===
> === <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name> <lock addr> (times locked)
> ===
> === Thread ID: 0x7f2083d11700 LWP:31458 (do_devstate_changes  started at [  621] devicestate.c ast_device_state_engine_init())
> === ---> Lock #0 (astobj2_container.c): MUTEX 310 internal_ao2_traverse self 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1abb982] asterisk astobj2_container.c:315 internal_ao2_traverse()
>         [0x5603f1abbd37] asterisk astobj2_container.c:421 __ao2_callback_data()
>         [0x5603f1afa0c8] asterisk channel.c:1275 ast_channel_callback()
>         [0x5603f1afa7f9] asterisk channel.c:1433 ast_channel_get_by_name_prefix()
>         [0x5603f1b42caa] asterisk devicestate.c:295 ast_parse_device_state()
>         [0x5603f1b4302b] asterisk devicestate.c:376 _ast_device_state()
>         [0x5603f1b434d1] asterisk devicestate.c:464 do_state_change()
>         [0x5603f1b438aa] asterisk devicestate.c:540 do_devstate_changes()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 1290 ast_channel_by_name_cb chan 0x7f206c4c0060 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1afa189] asterisk channel.c:1291 ast_channel_by_name_cb()
>         [0x5603f1abba0a] asterisk astobj2_container.c:326 internal_ao2_traverse()
>         [0x5603f1abbd37] asterisk astobj2_container.c:421 __ao2_callback_data()
>         [0x5603f1afa0c8] asterisk channel.c:1275 ast_channel_callback()
>         [0x5603f1afa7f9] asterisk channel.c:1433 ast_channel_get_by_name_prefix()
>         [0x5603f1b42caa] asterisk devicestate.c:295 ast_parse_device_state()
>         [0x5603f1b4302b] asterisk devicestate.c:376 _ast_device_state()
>         [0x5603f1b434d1] asterisk devicestate.c:464 do_state_change()
>         [0x5603f1b438aa] asterisk devicestate.c:540 do_devstate_changes()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: bridge_channel.c line 2879 (bridge_channel_internal_join)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067d04700 LWP:31499 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x5603f62ba660 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067c87700 LWP:31500 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2070ae46e0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067c0a700 LWP:31501 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f20614274c0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067b8d700 LWP:31502 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2078e8a4a0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067b10700 LWP:31503 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2076c18840 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067a93700 LWP:31504 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f207d5d6f10 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067a16700 LWP:31505 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2088b673f0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067999700 LWP:31506 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2085c8de80 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206791c700 LWP:31507 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f208cf92ef0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206789f700 LWP:31508 (iax2_process_thread  started at [12720] chan_iax2.c start_network_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x5603f748d6d0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f20697c4700 LWP:443 (worker_start         started at [ 1111] threadpool.c worker_thread_start())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f207c82fce0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f206bc30016] chan_pjsip.so chan_pjsip.c:563 chan_pjsip_new()
>         [0x7f206bc3ce0b] chan_pjsip.so chan_pjsip.c:3134 chan_pjsip_incoming_request()
>         [0x7f206bf0a4ed] res_pjsip_session.so res_pjsip_session.c:4418 handle_incoming_request()
>         [0x7f206bf07446] res_pjsip_session.so res_pjsip_session.c:4043 new_invite()
>         [0x7f206bf07cec] res_pjsip_session.so res_pjsip_session.c:4135 handle_new_invite_request()
>         [0x7f206bf08ee1] res_pjsip_session.so res_pjsip_session.c:4247 session_on_rx_request()
>         [0x7f20930f858f] libasteriskpj.so.2 sip_endpoint.c:930 pjsip_endpt_process_rx_data()
>         [0x7f206bfcc242] res_pjsip.so pjsip_distributor.c:956 distribute()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c4bd4e] asterisk threadpool.c:1354 execute_tasks()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c498b4] asterisk threadpool.c:367 threadpool_execute()
>         [0x5603f1c4b4f6] asterisk threadpool.c:1137 worker_active()
>         [0x5603f1c4b260] asterisk threadpool.c:1057 worker_start()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2067ffa700 LWP:3871 (worker_start         started at [ 1111] threadpool.c worker_thread_start())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f206d746c80 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f206bc30016] chan_pjsip.so chan_pjsip.c:563 chan_pjsip_new()
>         [0x7f206bc3ce0b] chan_pjsip.so chan_pjsip.c:3134 chan_pjsip_incoming_request()
>         [0x7f206bf0a4ed] res_pjsip_session.so res_pjsip_session.c:4418 handle_incoming_request()
>         [0x7f206bf07446] res_pjsip_session.so res_pjsip_session.c:4043 new_invite()
>         [0x7f206bf07cec] res_pjsip_session.so res_pjsip_session.c:4135 handle_new_invite_request()
>         [0x7f206bf08ee1] res_pjsip_session.so res_pjsip_session.c:4247 session_on_rx_request()
>         [0x7f20930f858f] libasteriskpj.so.2 sip_endpoint.c:930 pjsip_endpt_process_rx_data()
>         [0x7f206bfcc242] res_pjsip.so pjsip_distributor.c:956 distribute()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c4bd4e] asterisk threadpool.c:1354 execute_tasks()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c498b4] asterisk threadpool.c:367 threadpool_execute()
>         [0x5603f1c4b4f6] asterisk threadpool.c:1137 worker_active()
>         [0x5603f1c4b260] asterisk threadpool.c:1057 worker_start()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f209008c700 LWP:21787 (worker_start         started at [ 1111] threadpool.c worker_thread_start())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f207e9f3d30 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f206bc30016] chan_pjsip.so chan_pjsip.c:563 chan_pjsip_new()
>         [0x7f206bc3ce0b] chan_pjsip.so chan_pjsip.c:3134 chan_pjsip_incoming_request()
>         [0x7f206bf0a4ed] res_pjsip_session.so res_pjsip_session.c:4418 handle_incoming_request()
>         [0x7f206bf07446] res_pjsip_session.so res_pjsip_session.c:4043 new_invite()
>         [0x7f206bf07cec] res_pjsip_session.so res_pjsip_session.c:4135 handle_new_invite_request()
>         [0x7f206bf08ee1] res_pjsip_session.so res_pjsip_session.c:4247 session_on_rx_request()
>         [0x7f20930f858f] libasteriskpj.so.2 sip_endpoint.c:930 pjsip_endpt_process_rx_data()
>         [0x7f206bfcc242] res_pjsip.so pjsip_distributor.c:956 distribute()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c4bd4e] asterisk threadpool.c:1354 execute_tasks()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c498b4] asterisk threadpool.c:367 threadpool_execute()
>         [0x5603f1c4b4f6] asterisk threadpool.c:1137 worker_active()
>         [0x5603f1c4b260] asterisk threadpool.c:1057 worker_start()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f20650b1700 LWP:2130 (default_tps_processing_function started at [  226] taskprocessor.c default_listener_start())
> === ---> Waiting for Lock #0 (astobj2_container.c): MUTEX 308 internal_ao2_traverse self 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2065228700 LWP:6593 (pbx_thread           started at [ 4736] pbx.c ast_pbx_start())
> === ---> Lock #0 (bridge_channel.c): MUTEX 2877 bridge_channel_internal_join bridge_channel->bridge 0x7f206d6426c0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1ae3793] asterisk bridge.h:484 _ast_bridge_lock()
>         [0x5603f1aeb2c8] asterisk bridge_channel.c:2879 bridge_channel_internal_join()
>         [0x5603f1acca08] asterisk bridge.c:1725 ast_bridge_join()
>         [0x5603f1c9cf55] asterisk features.c:684 ast_bridge_call_with_flags()
>         [0x5603f1c9d113] asterisk features.c:715 ast_bridge_call()
>         [0x7f20812bd146] app_dial.so app_dial.c:3413 dial_exec_full()
>         [0x7f20812bda11] app_dial.so app_dial.c:3482 dial_exec()
>         [0x5603f1bbbc19] asterisk pbx_app.c:492 pbx_exec()
>         [0x5603f1ba42f6] asterisk pbx.c:2948 pbx_extension_helper()
>         [0x5603f1ba868f] asterisk pbx.c:4210 ast_spawn_extension()
>         [0x5603f1ba949f] asterisk pbx.c:4386 __ast_pbx_run()
>         [0x5603f1baae48] asterisk pbx.c:4711 pbx_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (bridge_channel.c): MUTEX 2879 bridge_channel_internal_join bridge_channel->chan 0x7f206d539fc0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1aeb2fb] asterisk bridge_channel.c:2881 bridge_channel_internal_join()
>         [0x5603f1acca08] asterisk bridge.c:1725 ast_bridge_join()
>         [0x5603f1c9cf55] asterisk features.c:684 ast_bridge_call_with_flags()
>         [0x5603f1c9d113] asterisk features.c:715 ast_bridge_call()
>         [0x7f20812bd146] app_dial.so app_dial.c:3413 dial_exec_full()
>         [0x7f20812bda11] app_dial.so app_dial.c:3482 dial_exec()
>         [0x5603f1bbbc19] asterisk pbx_app.c:492 pbx_exec()
>         [0x5603f1ba42f6] asterisk pbx.c:2948 pbx_extension_helper()
>         [0x5603f1ba868f] asterisk pbx.c:4210 ast_spawn_extension()
>         [0x5603f1ba949f] asterisk pbx.c:4386 __ast_pbx_run()
>         [0x5603f1baae48] asterisk pbx.c:4711 pbx_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (bridge_channel.c): MUTEX 2885 bridge_channel_internal_join peer 0x7f206c4c0060 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1afa189] asterisk channel.c:1291 ast_channel_by_name_cb()
>         [0x5603f1abba0a] asterisk astobj2_container.c:326 internal_ao2_traverse()
>         [0x5603f1abbd37] asterisk astobj2_container.c:421 __ao2_callback_data()
>         [0x5603f1afa0c8] asterisk channel.c:1275 ast_channel_callback()
>         [0x5603f1afa7f9] asterisk channel.c:1433 ast_channel_get_by_name_prefix()
>         [0x5603f1b42caa] asterisk devicestate.c:295 ast_parse_device_state()
>         [0x5603f1b4302b] asterisk devicestate.c:376 _ast_device_state()
>         [0x5603f1b434d1] asterisk devicestate.c:464 do_state_change()
>         [0x5603f1b438aa] asterisk devicestate.c:540 do_devstate_changes()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: bridge_channel.c line 2879 (bridge_channel_internal_join)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206541c700 LWP:6598 (bridge_channel_ind_thread started at [ 1892] bridge.c bridge_impart_internal())
> === ---> Waiting for Lock #0 (bridge_channel.c): MUTEX 235 ast_bridge_channel_lock_bridge bridge 0x7f206d6426c0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1ae3793] asterisk bridge.h:484 _ast_bridge_lock()
>         [0x5603f1ae3c77] asterisk bridge_channel.c:236 ast_bridge_channel_lock_bridge()
>         [0x5603f1ada376] asterisk bridge_basic.c:296 basic_hangup_hook()
>         [0x5603f1ae50b3] asterisk bridge_channel.c:613 ast_bridge_channel_kick()
>         [0x5603f1aea8b9] asterisk bridge_channel.c:2595 bridge_handle_trip()
>         [0x5603f1aeaf78] asterisk bridge_channel.c:2803 bridge_channel_wait()
>         [0x5603f1aeb9de] asterisk bridge_channel.c:2975 bridge_channel_internal_join()
>         [0x5603f1accdd5] asterisk bridge.c:1798 bridge_channel_ind_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: bridge_channel.c line 2877 (bridge_channel_internal_join)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f20669fa700 LWP:6599 (bridge_channel_ind_thread started at [ 1892] bridge.c bridge_impart_internal())
> === ---> Lock #0 (bridge_channel.c): MUTEX 2877 bridge_channel_internal_join bridge_channel->bridge 0x7f208d2dcef0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1ae3793] asterisk bridge.h:484 _ast_bridge_lock()
>         [0x5603f1aeb2c8] asterisk bridge_channel.c:2879 bridge_channel_internal_join()
>         [0x5603f1accdd5] asterisk bridge.c:1798 bridge_channel_ind_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (bridge_channel.c): MUTEX 2879 bridge_channel_internal_join bridge_channel->chan 0x7f206c4c0060 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1aeb2fb] asterisk bridge_channel.c:2881 bridge_channel_internal_join()
>         [0x5603f1accdd5] asterisk bridge.c:1798 bridge_channel_ind_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (bridge_channel.c): MUTEX 2885 bridge_channel_internal_join peer 0x7f206d539fc0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1aeb349] asterisk bridge_channel.c:2886 bridge_channel_internal_join()
>         [0x5603f1accdd5] asterisk bridge.c:1798 bridge_channel_ind_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: bridge_channel.c line 2879 (bridge_channel_internal_join)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2066082700 LWP:6632 (worker_start         started at [ 1111] threadpool.c worker_thread_start())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f207deb3e30 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f206bc30016] chan_pjsip.so chan_pjsip.c:563 chan_pjsip_new()
>         [0x7f206bc3ce0b] chan_pjsip.so chan_pjsip.c:3134 chan_pjsip_incoming_request()
>         [0x7f206bf0a4ed] res_pjsip_session.so res_pjsip_session.c:4418 handle_incoming_request()
>         [0x7f206bf07446] res_pjsip_session.so res_pjsip_session.c:4043 new_invite()
>         [0x7f206bf07cec] res_pjsip_session.so res_pjsip_session.c:4135 handle_new_invite_request()
>         [0x7f206bf08ee1] res_pjsip_session.so res_pjsip_session.c:4247 session_on_rx_request()
>         [0x7f20930f858f] libasteriskpj.so.2 sip_endpoint.c:930 pjsip_endpt_process_rx_data()
>         [0x7f206bfcc242] res_pjsip.so pjsip_distributor.c:956 distribute()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c4bd4e] asterisk threadpool.c:1354 execute_tasks()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c498b4] asterisk threadpool.c:367 threadpool_execute()
>         [0x5603f1c4b4f6] asterisk threadpool.c:1137 worker_active()
>         [0x5603f1c4b260] asterisk threadpool.c:1057 worker_start()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2069747700 LWP:6633 (worker_start         started at [ 1111] threadpool.c worker_thread_start())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2088ca6da0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f206bc30016] chan_pjsip.so chan_pjsip.c:563 chan_pjsip_new()
>         [0x7f206bc3ce0b] chan_pjsip.so chan_pjsip.c:3134 chan_pjsip_incoming_request()
>         [0x7f206bf0a4ed] res_pjsip_session.so res_pjsip_session.c:4418 handle_incoming_request()
>         [0x7f206bf07446] res_pjsip_session.so res_pjsip_session.c:4043 new_invite()
>         [0x7f206bf07cec] res_pjsip_session.so res_pjsip_session.c:4135 handle_new_invite_request()
>         [0x7f206bf08ee1] res_pjsip_session.so res_pjsip_session.c:4247 session_on_rx_request()
>         [0x7f20930f858f] libasteriskpj.so.2 sip_endpoint.c:930 pjsip_endpt_process_rx_data()
>         [0x7f206bfcc242] res_pjsip.so pjsip_distributor.c:956 distribute()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c4bd4e] asterisk threadpool.c:1354 execute_tasks()
>         [0x5603f1c41f6c] asterisk taskprocessor.c:1239 ast_taskprocessor_execute()
>         [0x5603f1c498b4] asterisk threadpool.c:367 threadpool_execute()
>         [0x5603f1c4b4f6] asterisk threadpool.c:1137 worker_active()
>         [0x5603f1c4b260] asterisk threadpool.c:1057 worker_start()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f20643ce700 LWP:7806 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f207f2d2898 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb61c4] asterisk pbx.c:8444 ast_pbx_outgoing_exten_predial()
>         [0x5603f1bb6129] asterisk pbx.c:8423 ast_pbx_outgoing_exten()
>         [0x7f2083a3e052] pbx_spool.so pbx_spool.c:443 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f207d0d3590 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb61c4] asterisk pbx.c:8444 ast_pbx_outgoing_exten_predial()
>         [0x5603f1bb6129] asterisk pbx.c:8423 ast_pbx_outgoing_exten()
>         [0x7f2083a3e052] pbx_spool.so pbx_spool.c:443 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2065516700 LWP:10658 (iax2_process_thread  started at [ 1626] chan_iax2.c find_idle_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f208d1bf620 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206670c700 LWP:10667 (iax2_process_thread  started at [ 1626] chan_iax2.c find_idle_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2070ac7120 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206668f700 LWP:10689 (iax2_process_thread  started at [ 1626] chan_iax2.c find_idle_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2078eb8cf0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f20680f4700 LWP:10690 (iax2_process_thread  started at [ 1626] chan_iax2.c find_idle_thread())
> === ---> Lock #0 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f205ca4c040 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x7f2080fbdbbb] chan_iax2.so chan_iax2.c:5901 ast_iax2_new()
>         [0x7f2080fdfbf8] chan_iax2.so chan_iax2.c:11493 socket_process_helper()
>         [0x7f2080fe519b] chan_iax2.so chan_iax2.c:12073 socket_process()
>         [0x7f2080fe5851] chan_iax2.so chan_iax2.c:12184 iax2_process_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #1 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206444b700 LWP:20245 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f205647e9f8 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f20567c5740 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2064fb7700 LWP:20580 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x5603f73717a8 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x5603f6d92510 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2066883700 LWP:20849 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f2078e3ef88 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2078dfb650 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206641e700 LWP:21042 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f208886ffa8 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2088b7cde0 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2066324700 LWP:21379 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f2070621b68 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2070bfce20 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2066518700 LWP:21625 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f205735b548 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f20574cfa00 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2066789700 LWP:21839 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f2053cd3658 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2048f8ac00 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f206622a700 LWP:22010 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f2070b2b8c8 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f2070bf6880 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2065f88700 LWP:22282 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x7f207cf88398 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x7f207cc72a50 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> === Thread ID: 0x7f2064ebd700 LWP:22553 (attempt_thread       started at [  475] pbx_spool.c launch_service())
> === ---> Lock #0 (dial.c): MUTEX 439 ast_dial_prerun &(&dial->channels)->lock 0x5603f7a5bf08 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1b4581f] asterisk dial.c:440 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Lock #1 (channel.c): MUTEX 920 __ast_channel_alloc_ap tmp 0x5603f6dbfa10 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c3f] asterisk channel.c:922 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === ---> Waiting for Lock #2 (channel.c): MUTEX 922 __ast_channel_alloc_ap channels 0x5603f3b4e080 (1)
>         [0x5603f1b81517] asterisk lock.c:272 __ast_pthread_mutex_lock()
>         [0x5603f1ab9091] asterisk astobj2.c:241 __ao2_lock()
>         [0x5603f1af8c6e] asterisk channel.c:924 __ast_channel_alloc_ap()
>         [0x5603f1af8f56] asterisk channel.c:966 __ast_channel_alloc()
>         [0x5603f1b3d14c] asterisk core_unreal.c:1204 ast_unreal_new_channels()
>         [0x5603f1b39195] asterisk core_local.c:1027 local_request_with_stream_topology()
>         [0x5603f1b0acfa] asterisk channel.c:6347 request_channel()
>         [0x5603f1b0b3ca] asterisk channel.c:6442 ast_request()
>         [0x5603f1b45352] asterisk dial.c:368 begin_dial_prerun()
>         [0x5603f1b45845] asterisk dial.c:441 ast_dial_prerun()
>         [0x5603f1bb595e] asterisk pbx.c:8264 pbx_outgoing_attempt()
>         [0x5603f1bb64a6] asterisk pbx.c:8507 ast_pbx_outgoing_app_predial()
>         [0x5603f1bb63e7] asterisk pbx.c:8487 ast_pbx_outgoing_app()
>         [0x7f2083a3df24] pbx_spool.so pbx_spool.c:437 attempt_thread()
>         [0x5603f1c57820] asterisk utils.c:1572 dummy_start()
>         [0x7f209292afa3] libpthread.so.0 pthread_create.c:487 start_thread()
>         [0x7f20923beeff] libc.so.6 clone.S:97 clone()
> === --- ---> Locked Here: astobj2_container.c line 310 (internal_ao2_traverse)
> === -------------------------------------------------------------------
> ===
> =======================================================================
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)



More information about the asterisk-bugs mailing list