[asterisk-bugs] [JIRA] (ASTERISK-26986) Poor high load perfomance, locks, lost rtp

Joshua Colp (JIRA) noreply at issues.asterisk.org
Wed May 10 08:24:58 CDT 2017


     [ https://issues.asterisk.org/jira/browse/ASTERISK-26986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joshua Colp updated ASTERISK-26986:
-----------------------------------

    Assignee: Badalian Vyacheslav
      Status: Waiting for Feedback  (was: Triage)

> Poor high load perfomance, locks,lost rtp
> -----------------------------------------
>
>                 Key: ASTERISK-26986
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-26986
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>    Affects Versions: 13.15.0
>            Reporter: Badalian Vyacheslav
>            Assignee: Badalian Vyacheslav
>            Severity: Minor
>
> Have
> 32 core system with 256 GB memory. Running only asterisk.
> 500 concurrent calls to MOH - 1 avg, 100%-150% cpu.(of 3200%)
> running test by 
> {noformat}
> sipp -sn uac  -d 10000 -r 50 -s 1001 127.0.0.1 -l 2000 -p 8877 -mi 127.0.0.1
> [sipp]
> type=friend
> context=sipp
> host=dynamic
> port=6000
> user=sipp
> canreinvite=no
> disallow=all
> allow=alaw
> allow=ulaw
> transport=udp
> [sipp]
> exten => 1001,1,Answer
> exten => 1001,n,MusicOnHold(default)
> exten => 1001,n,Wait(20)
> exten => 1001,n,Hangup
> {noformat}
> In simple load test on sipp have many perfomnce issues:
> 1. FRACK!
> {noformat}
> [2017-05-06 05:48:29] ERROR[32005][C-000152e7]: astobj2.c:518 __ao2_ref: Excessive refcount 100000 reached on ao2 object 0x6080015d92f0
> [2017-05-06 05:48:29] ERROR[32005][C-000152e7]: astobj2.c:518 __ao2_ref: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x6080015d92f0 (0)
> Got 29 backtrace records
> #0: [0x67e4d4] main/logger.c:1979 ast_log_backtrace() (0x67e4c0+14)
> #1: [0x7fe347] main/utils.c:2477 __ast_assert_failed() (0x7fe298+AF)
> #2: [0x496162] main/astobj2.c:437 internal_ao2_ref()
> #3: [0x4965d0] main/astobj2.c:519 __ao2_ref() (0x4965a2+2E)
> #4: [0x7a7206] main/stasis_message.c:125 stasis_message_create_full() (0x7a716d+99)
> #5: [0x7a73d4] main/stasis_message.c:139 stasis_message_create() (0x7a73ac+28)
> #6: [0x795c47] main/stasis_cache.c:789 update_create()
> #7: [0x795efd] main/stasis_cache.c:839 caching_topic_exec()
> #8: [0x783d4f] main/stasis.c:437 subscription_invoke()
> #9: [0x785c12] main/stasis.c:768 dispatch_message()
> #10: [0x786031] main/stasis.c:830 publish_msg()
> #11: [0x7860ca] main/stasis.c:844 stasis_publish() (0x7860a2+28)
> #12: [0x7a0370] main/stasis_channels.c:727 ast_channel_publish_snapshot() (0x7a014f+221)
> #13: [0x7a00a0] main/stasis_channels.c:701 ast_channel_stage_snapshot_done() (0x79ff4e+152)
> #14: [0x54c31e] main/channel.c:1004 __ast_channel_alloc_ap()
> #15: [0x54c4bd] main/channel.c:1019 __ast_channel_alloc() (0x54c378+145)
> #16: [0x7fbb94001ade] channels/chan_sip.c:8108 sip_new()
> #17: [0x7fbb940b5038] channels/chan_sip.c:26408 handle_request_invite()
> #18: [0x7fbb940c9a4e] channels/chan_sip.c:28835 handle_incoming()
> #19: [0x7fbb940cb1ec] channels/chan_sip.c:29044 handle_request_do()
> #20: [0x7fbb940ca6c4] channels/chan_sip.c:28976 sipsock_read()
> #21: [0x65efed] main/io.c:295 ast_io_wait() (0x65ebb1+43C)
> #22: [0x7fbb940cf045] channels/chan_sip.c:29610 do_monitor()
> #23: [0x7f7683] main/utils.c:1235 dummy_start()
> {noformat}
> 2. Periodic rtp issue ( load avg <1. asterisk cpu load 200%) 
> {noformat}
> [2017-05-06 05:31:04] NOTICE[32005]: chan_sip.c:29516 check_rtp_timeout: Disconnecting call 'SIP/sipp-000080bc' for lack of RTP activity in 31 seconds
> {noformat}
> 3. Issues with taskprocessors
> {noformat}
> [2017-05-06 05:30:32] WARNING[65015][C-00008084]: taskprocessor.c:888 taskprocessor_push: The 'subm:manager_topic-00000007' task processor queue reached 3000 scheduled tasks.
> [2017-05-06 05:30:32] WARNING[64764][C-00007f89]: taskprocessor.c:888 taskprocessor_push: The 'subm:ast_channel_topic_all-cached-00008b4e' task processor queue reached 500 scheduled tasks.
> [2017-05-06 05:30:32] WARNING[64760][C-00007f85]: taskprocessor.c:888 taskprocessor_push: The 'subm:cel_aggregation_topic-00000006' task processor queue reached 3000 scheduled tasks.
> [2017-05-06 05:30:32] WARNING[65489]: taskprocessor.c:888 taskprocessor_push: The 'subp:SIP/sipp-00008b48' task processor queue reached 500 scheduled tasks.
> [2017-05-06 05:30:32] WARNING[64938][C-00008037]: taskprocessor.c:888 taskprocessor_push: The 'subm:cdr_engine-00000003' task processor queue reached 5000 scheduled tasks.
> m1-asterisk01t*CLI> core show taskprocessors
> Processor                                      Processed   In Queue  Max Depth  Low water High water
> app_voicemail                                          0          0          0        450        500
> ast_msg_queue                                          0          0          0        450        500
> CCSS_core                                              0          0          0        450        500
> hep_queue_tp                                           0          0          0        450        500
> iax2_transmit                                          0          0          0        450        500
> pjsip/default-0000000c                                 4          0          1        450        500
> pjsip/default-0000000d                                 3          0          1        450        500
> pjsip/default-0000000e                                 3          0          1        450        500
> pjsip/default-0000000f                                 3          0          1        450        500
> pjsip/default-00000010                                 3          0          1        450        500
> pjsip/default-00000011                                 3          0          1        450        500
> pjsip/default-00000012                                 3          0          1        450        500
> pjsip/default-00000013                                 3          0          1        450        500
> pjsip/distributor-0000001d                             0          0          0        450        500
> pjsip/distributor-0000001e                             0          0          0        450        500
> pjsip/distributor-0000001f                             0          0          0        450        500
> pjsip/distributor-00000020                             0          0          0        450        500
> pjsip/distributor-00000021                             0          0          0        450        500
> pjsip/distributor-00000022                             0          0          0        450        500
> pjsip/distributor-00000023                             0          0          0        450        500
> pjsip/distributor-00000024                             0          0          0        450        500
> pjsip/distributor-00000025                             0          0          0        450        500
> pjsip/distributor-00000026                             0          0          0        450        500
> pjsip/distributor-00000027                             0          0          0        450        500
> pjsip/distributor-00000028                             0          0          0        450        500
> pjsip/distributor-00000029                             0          0          0        450        500
> pjsip/distributor-0000002a                             0          0          0        450        500
> pjsip/distributor-0000002b                             0          0          0        450        500
> pjsip/distributor-0000002c                             0          0          0        450        500
> pjsip/distributor-0000002d                             0          0          0        450        500
> pjsip/distributor-0000002e                             0          0          0        450        500
> pjsip/distributor-0000002f                             0          0          0        450        500
> pjsip/distributor-00000030                             0          0          0        450        500
> pjsip/distributor-00000031                             0          0          0        450        500
> pjsip/distributor-00000032                             0          0          0        450        500
> pjsip/distributor-00000033                             0          0          0        450        500
> pjsip/distributor-00000034                             0          0          0        450        500
> pjsip/distributor-00000035                             0          0          0        450        500
> pjsip/distributor-00000036                             0          0          0        450        500
> pjsip/distributor-00000037                             0          0          0        450        500
> pjsip/distributor-00000038                             0          0          0        450        500
> pjsip/distributor-00000039                             0          0          0        450        500
> pjsip/distributor-0000003a                             0          0          0        450        500
> pjsip/distributor-0000003b                             0          0          0        450        500
> pjsip/messaging                                        0          0          0        450        500
> pjsip/mwi-00000042                                     0          0          0        450        500
> pjsip/mwi-00000043                                     0          0          0        450        500
> pjsip/mwi-00000044                                     0          0          0        450        500
> pjsip/mwi-00000045                                     0          0          0        450        500
> pjsip/mwi-00000046                                     0          0          0        450        500
> pjsip/mwi-00000047                                     0          0          0        450        500
> pjsip/mwi-00000048                                     0          0          0        450        500
> pjsip/mwi-00000049                                     0          0          0        450        500
> SIP                                                   25          0          1        450        500
> SIP-control                                           56          0          2        450        500
> Sorcery                                               10          0          4        450        500
> Sorcery-control                                       25          0          4        450        500
> sorcery/acl-00008b49                                   0          0          0        450        500
> sorcery/aor-00000019                                   1          0          1        450        500
> sorcery/asterisk-publication-0000004b                  0          0          0        450        500
> sorcery/auth-00000014                                  0          0          0        450        500
> sorcery/bucket-00000000                                0          0          0        450        500
> sorcery/contact-00000018                               1          0          1       1350       1500
> sorcery/contact_status-0000001a                        1          0          1       1350       1500
> sorcery/domain_alias-0000001b                          0          0          0        450        500
> sorcery/endpoint-00000015                              1          0          1        450        500
> sorcery/file-00000001                                  0          0          0        450        500
> sorcery/global-0000001c                                5          0          1        450        500
> sorcery/identify-00000041                              0          0          0        450        500
> sorcery/inbound-publication-0000003f                   0          0          0        450        500
> sorcery/log_mappings-0000000a                          0          0          0        450        500
> sorcery/nat_hook-00000016                              0          0          0        450        500
> sorcery/outbound-publish-0000003c                      0          0          0        450        500
> sorcery/phoneprov-00008b4a                             0          0          0        450        500
> sorcery/registration-00008b4b                          1          0          1        450        500
> sorcery/resource_list-0000003e                         0          0          0        450        500
> sorcery/subscription_persistence-0000003d              0          0          0        450        500
> sorcery/system-0000000b                                0          0          0        450        500
> sorcery/transport-00000017                             0          0          0        450        500
> stasis-core                                      2023100          1          9        450        500
> stasis-core-control                              4046215          1         17        450        500
> subm:ast_bridge_topic_all-cached-00008b4d              1          0          1        450        500
> subm:ast_channel_topic_all-00008b51              1369195          0        258        450        500
> subm:ast_channel_topic_all-cached-00008b4e       1986136          0        556        450        500
> subm:ast_device_state_topic-00000002              205492          0          4        450        500
> subm:ast_device_state_topic-00000004              205491          0          4        450        500
> subm:ast_device_state_topic-00008b50              205484          0         14        450        500
> subm:ast_presence_state_topic_all-00000005             1          0          1        450        500
> subm:ast_security-00008b4f                             2          0          1        450        500
> subm:ast_system-00000009                              35          0         29        450        500
> subm:ast_system-0000004c                              34          0          1        450        500
> subm:ast_system-0000004d                              33          0          1        450        500
> subm:ast_system-00008b4c                               1          0          1        450        500
> subm:cdr_engine-00000003                         1986136          1       7438       4500       5000
> subm:cel_aggregation_topic-00000006              1986134          3       7475       2700       3000
> subm:endpoint_topic_all-cached-00000008          2366516          0        458        450        500
> subm:manager_topic-00000007                      2042433     217520     218830       2700       3000
> ...
> subp:SIP/sipp-00008b48                           2124754          0       6451        450        500
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)



More information about the asterisk-bugs mailing list