[asterisk-bugs] [JIRA] (ASTERISK-28625) Playback of local files impacted by large media cache

[Kevin Reeves (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=248852#comment-248852 ] 

Kevin Reeves commented on ASTERISK-28625:
-----------------------------------------

Yes - we're using the media cache to Playback audio files from an http address.

I'm unclear about why this seems to be impacting performance.  The hash table should be fast - which you proved with 25k items in your development environment and you're not seeing any issues.  So I don't know what impact load has on the hash searching operations.  Maybe under heavy load we introduce a contention issue for the hash table?  

Maybe when new items are coming into the media cache and it's syncing to the astdb it locks the container just long enough to block other requests for it?

What's interesting about that question specifically is that when we were experiencing these issues and our script was trying to delete items from the media cache it was going really slow on each delete operation.  I see in the code in ast_media_cache_delete() and ast_media_cache_retrieve() that it's writing to the astdb as well.  So I wonder what the experience is with other requests to search the media cache while it's writing to the db?

Not sure - just brainstorming.

But with regards to my suggested fix - I am suggesting that only remote files should be using the media cache.  Otherwise, we shouldn't be searching that hash table at all.  Or is that a bad assumption?

> Playback of local files impacted by large media cache
> -----------------------------------------------------
>
>                 Key: ASTERISK-28625
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28625
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Core/Streams
>    Affects Versions: 16.6.1
>         Environment: CentOS Linux release 7.6.1810
>            Reporter: Kevin Reeves
>            Assignee: Kevin Reeves
>              Labels: patch
>         Attachments: ASTERISK-28625.patch
>
>
> Playback of local files is impacted by media cache.  As the cache grows, performance of common Read and Playback (and presumably other) operations for local files is impacted.
> For example, Playback command of a local file like this:
> exten => s,n,Playback(thankyou)
> Even though it's playing a local file, those operations are searching the media cache. As the cache size grows, the performance of that type of call is impacted.  In fact, we've seen a multi-second delay from the time the Playback command is executed to when the caller hears the audio.
> The delay grows significantly over time as the cache grows bigger.  With thousands of files in the media cache while the server is under heavy load, the delay has been upwards of 15-30 seconds.  Without files in the media cache there is zero delay.
> Here's a debug output snippet that shows the issue.  The Read application is playing a local file named 'pb/ac-enter-pin'.  Notice that media_cache.c line 194 fails to locate the media.
>     -- Executing [s at nx-dial-session:20] Read("SIP/vitel-inbound2-00000002", "__DS_PIN,/var/lib/asterisk/sounds/pb/ac-enter-pin") in new stack
> [2019-11-20 15:22:06] DEBUG[30329][C-00000003]: media_cache.c:194 ast_media_cache_retrieve: Failed to obtain media at '/var/lib/asterisk/sounds/pb/ac-enter-pin'
> [2019-11-20 15:22:06] DEBUG[30329][C-00000003]: media_cache.c:194 ast_media_cache_retrieve: Failed to obtain media at '/var/lib/asterisk/sounds/pb/ac-enter-pin'
> This section of the code should not be used in cases where the file is a local file.



--
This message was sent by Atlassian JIRA
(v6.2#6252)