[asterisk-dev] Review Request: Sorcery Caching

Tue Apr 28 12:44:59 CDT 2015

On the subject of maxiumum cache size - I like being able to limit the
memory usage, but there is apparently no way to prioritize often used
entries.  I'm thinking of an example where my cache is full, having gone
through every entry, but then a small subset of entries is frequently read
which may not have been the first into the cache.  If I understand the
wording of the wiki article correctly, these would then not benefit from
caching, since they would not be allowed in the cache?

Also, I'm unclear on the differences between the three object
expire/lifetime values.  Can you add more descriptive details to each of
these values, possibly give an example showing why it is beneficial?

My thoughts on cache needs are this:

1) I want to be able to set the maximum memory dedicated to caching, either
in MB or in # of entries, but have the caching algorithm take into account
the most recently read entries when determining what to hold onto and what
to throw away.  Something I've just looked up is more likely to used again
than something that has been sitting in the cache for a while.  To
accomplish this each entry needs a timestamp of the last cache hit (when it
was last used/read).

2) I want to be able to set two maximum time in cache values: The first is
the seconds after which the entry is stale, and the second where it is
guarunteed to be removed from the cache and no longer used.  Both values
are from the time the entry was last obtained from the source (db),
requiring a timestamp on entry creation or when it was last refreshed from
the source.  Before the entry is stale, it can be re-used without any
further action needed.  When a stale entry is accessed (the first time) it
should trigger a background refresh to ensure that changes have not been
made, but continue to return a hit from the cache until the second
timeout.  Background refreshes to be initiated by a separate thread only
when there are no primary requests active.  The idea here is I can set 300
seconds for stale, and 600 for lifetime -- for a frequently used entry I'm
only refreshing from the database every 2.5 minutes and 100% of the
requests come immediately from the cache with no database lookup delay to
the requesting thread.  For any entry that hasn't been read in 5 minutes,
it's deleted so the memory utilization goes away also.

3) Flushing all entries on a periodic interval doesn't sound like an option
I'd be interested in -- assuming that a maximum time in cache is enforced
properly, this seems like an odd duplication.  My worry is that dumping all
the entries will occasionally occur in the middle of a burst of reads, and
thus significantly slow things down.

4) The external notification should be able to forcibly empty the cache, or
optionally instead just force all objects to be stale without removing
them.  This would require adding a stale flag to the entry.

5) The rebalancing option sounds like a good idea, but since it is such an
implementation specific value, it would be useful to run some tests and
document some statistics on what conditions are needed for it to go from
negligable to noticeable improvement, so that people have an idea of when
and how to use it.

On Tue, Apr 28, 2015 at 11:28 AM, Joshua Colp <jcolp at digium.com> wrote:

> Kia ora,
>
> I've created a wiki page[1] which details the beginnings of a basic memory
> based caching wizard for sorcery. Right now while caching is possible using
> the existing memory wizard it's not possible to define object lifetimes, so
> once cached it's always pulled from the cache. This wiki page uses the
> memory wizard as a base but defines options which can tweak the behavior.
> Going forward this could serve as a basis for other wizards to be created
> for caching purposes.
>
> Some things to consider:
> 1. How much control and flexibility should we allow?
> 2. Are there additional mechanisms that should be exposed to allow
> explicit object expiration?
> 3. Are the defaults sane?
> 4. Is there additional testing that should be done?
> 5. Does anything need additional explanation?
>
> Cheers,
>
> [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching
>
> --
> Joshua Colp
> Digium, Inc. | Senior Software Developer
> 445 Jan Davis Drive NW - Huntsville, AL 35806 - US
> Check us out at: www.digium.com & www.asterisk.org
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-dev mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-dev
>

-- 
[image: Digium logo]
Scott Griepentrog
Digium, Inc · Software Developer
445 Jan Davis Drive NW · Huntsville, AL 35806 · US
direct/fax: +1 256 428 6239 · mobile: +1 256 580 6090
Check us out at: http://digium.com · http://asterisk.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20150428/01a5a988/attachment.html>