[asterisk-commits] media cache: Add a core API and facade for a backend agnosti... (asterisk[master])
SVN commits to the Asterisk project
asterisk-commits at lists.digium.com
Thu Jul 16 20:41:05 CDT 2015
Matt Jordan has submitted this change and it was merged.
Change subject: media cache: Add a core API and facade for a backend agnostic media cache
......................................................................
media cache: Add a core API and facade for a backend agnostic media cache
This patch adds a new API to the Asterisk core that acts as a media
cache. The core API itself is mostly a thin wrapper around some bucket
API provided implementation that itself acts as the mechanism of
retrieval for media. The media cache API in the core provides the
following:
* A very thin in-memory cache of the active bucket_file items. Unlike a
more traditional cache, it provides no expiration mechanisms. Most
queries that hit the in-memory cache will also call into the bucket
implementations as well. The bucket implementations are responsible
for determining whether or not the active record is active and valid.
This makes sense for the most likely implementation of a media cache
backend, i.e., HTTP. The HTTP layer itself is the actual arbiter of
whether or not a record is truly active; as such, the in-memory cache
in the core has to defer to it.
* The ability to create new items in the media cache from local
resources. This allows for re-creation of items in the cache on
restart.
* Synchronization of items in the media cache to the AstDB. This
also includes various pieces of important metadata.
The API provides sufficient access that higher level APIs, such as the
file or app APIs, do not have to worry about the semantics of the bucket
APIs when needing to playback a resource.
In addition, this patch provides unit tests for the media cache API. The
unit tests use a fake bucket backend to verify correctness.
Change-Id: I11227abbf14d8929eeb140ddd101dd5c3820391e
---
M include/asterisk/bucket.h
A include/asterisk/media_cache.h
M main/asterisk.c
M main/bucket.c
A main/media_cache.c
A tests/test_media_cache.c
6 files changed, 1,094 insertions(+), 3 deletions(-)
Approvals:
Anonymous Coward #1000019: Verified
Matt Jordan: Looks good to me, approved
Joshua Colp: Looks good to me, but someone else must approve
diff --git a/include/asterisk/bucket.h b/include/asterisk/bucket.h
index 4a27c3c..90d976a 100644
--- a/include/asterisk/bucket.h
+++ b/include/asterisk/bucket.h
@@ -23,7 +23,7 @@
*/
/*!
- * \page AstBucket Bucket File API
+ * \page bucket AstBucket Bucket File API
*
* Bucket is an API which provides directory and file access in a generic fashion. It is
* implemented as a thin wrapper over the sorcery data access layer API and is written in
diff --git a/include/asterisk/media_cache.h b/include/asterisk/media_cache.h
new file mode 100644
index 0000000..f1618b8
--- /dev/null
+++ b/include/asterisk/media_cache.h
@@ -0,0 +1,175 @@
+/*
+ * Asterisk -- An open source telephony toolkit.
+ *
+ * Copyright (C) 2015, Digium, Inc.
+ *
+ * Matt Jordan <mjordan at digium.com>
+ *
+ * See http://www.asterisk.org for more information about
+ * the Asterisk project. Please do not directly contact
+ * any of the maintainers of this project for assistance;
+ * the project provides a web site, mailing lists and IRC
+ * channels for your use.
+ *
+ * This program is free software, distributed under the terms of
+ * the GNU General Public License Version 2. See the LICENSE file
+ * at the top of the source tree.
+ */
+
+/*!
+ * \file
+ * \brief An in-memory media cache
+ */
+
+#ifndef _ASTERISK_MEDIA_CACHE_H
+#define _ASTERISK_MEDIA_CACHE_H
+
+#if defined(__cplusplus) || defined(c_plusplus)
+extern "C" {
+#endif
+
+struct ast_variable;
+
+/*!
+ * \brief Check if an item exists in the cache
+ *
+ * \param uri The unique URI for the media item
+ *
+ * \retval 0 uri does not exist in cache
+ * \retval 1 uri does exist in cache
+ */
+int ast_media_cache_exists(const char *uri);
+
+/*!
+ * \brief Retrieve an item from the cache
+ *
+ * \param uri The unique URI for the media item
+ * \param preferred_file_name The preferred name for the file storing the
+ * media once it is retrieved. Can be NULL.
+ * \param file_path Buffer to store the full path to the media in the
+ * cache
+ * \param len The length of the buffer pointed to by \c file_path
+ *
+ * \retval 0 The item was retrieved successfully
+ * \retval -1 The item could not be retrieved
+ *
+ * Example Usage:
+ * \code
+ * char media[PATH_MAX];
+ * int res;
+ *
+ * res = ast_media_cache_retrieve("http://localhost/foo.wav", NULL,
+ * media, sizeof(media));
+ * \endcode
+ *
+ * \details
+ * Retrieving an item will cause the \ref bucket Bucket backend associated
+ * with the URI scheme in \c uri to be queried. If the Bucket backend
+ * does not require an update, the cached information is used to find the
+ * file associated with \c uri, and \c file_path is populated with the
+ * location of the media file associated with \c uri.
+ *
+ * If the item is not in the cache, the item will be retrieved using the
+ * \ref bucket backend. When this occurs, if \c preferred_file_name is given,
+ * it will be used as the destination file for the retrieval. When retrieval
+ * of the media from the backend is complete, \c file_path is then populated
+ * as before.
+ */
+int ast_media_cache_retrieve(const char *uri, const char *preferred_file_name,
+ char *file_path, size_t len);
+
+/*!
+ * \brief Retrieve metadata from an item in the cache
+ *
+ * \param uri The unique URI for the media item
+ * \param key The key of the metadata to retrieve
+ * \param value Buffer to store the value in
+ * \param len The length of the buffer pointed to by \c value
+ *
+ * \retval 0 The metadata was retrieved successfully
+ * \retval -1 The metadata could not be retrieved
+ *
+ * Example Usage:
+ * \code
+ *
+ * int res;
+ * char file_size[32];
+ *
+ * res = ast_media_cache_retrieve_metadata("http://localhost/foo.wav", "size",
+ * file_size, sizeof(file_size));
+ * \endcode
+ */
+int ast_media_cache_retrieve_metadata(const char *uri, const char *key,
+ char *value, size_t len);
+
+/*!
+ * \brief Create/update a cached media item
+ *
+ * \param uri The unique URI for the media item to store in the cache
+ * \param file_path Full path to the media file to be cached
+ * \param metadata Metadata to store with the cached item
+ *
+ * \retval 0 The item was cached
+ * \retval -1 An error occurred when creating/updating the item
+ *
+ * Example Usage:
+ * \code
+ * int res;
+ *
+ * res = ast_media_cache_create_or_update("http://localhost/foo.wav",
+ * "/tmp/foo.wav", NULL);
+ * \endcode
+ *
+ * \note This method will overwrite whatever has been provided by the
+ * \ref bucket backend.
+ *
+ * \details
+ * While \ref ast_media_cache_retrieve is used to retrieve media from
+ * some \ref bucket provider, this method allows for overwriting what
+ * is provided by a backend with some local media. This is useful for
+ * reconstructing or otherwise associating local media with a remote
+ * URI, deferring updating of the media from the backend to some later
+ * retrieval.
+ */
+int ast_media_cache_create_or_update(const char *uri, const char *file_path,
+ struct ast_variable *metadata);
+
+/*!
+ * \brief Remove an item from the media cache
+ *
+ * \param uri The unique URI for the media item to store in the cache
+ *
+ * \retval 0 success
+ * \retval -1 error
+ *
+ * Example Usage:
+ * \code
+ * int res;
+ *
+ * res = ast_media_cache_delete("http://localhost/foo.wav");
+ * \endcode
+ *
+ * \details
+ * This removes an item completely from the media cache. Any files local
+ * on disk associated with the item are deleted as well.
+ *
+ * \note It is up to the \ref bucket implementation whether or not this
+ * affects any non-local storage
+ */
+int ast_media_cache_delete(const char *uri);
+
+/*!
+ * \brief Initialize the media cache
+ *
+ * \note This should only be called once, during Asterisk initialization
+ *
+ * \retval 0 success
+ * \retval -1 error
+ */
+int ast_media_cache_init(void);
+
+#if defined(__cplusplus) || defined(c_plusplus)
+}
+#endif
+
+#endif /* _ASTERISK_MEDIA_CACHE_H */
diff --git a/main/asterisk.c b/main/asterisk.c
index 0478f6c..4660bf9 100644
--- a/main/asterisk.c
+++ b/main/asterisk.c
@@ -248,6 +248,7 @@
#include "asterisk/endpoints.h"
#include "asterisk/codec.h"
#include "asterisk/format_cache.h"
+#include "asterisk/media_cache.h"
#include "../defaults.h"
@@ -4606,6 +4607,16 @@
exit(moduleresult == -2 ? 2 : 1);
}
+ /*
+ * This has to load after the dynamic modules load, as items in the media
+ * cache can't be constructed from items in the AstDB without their
+ * bucket backends.
+ */
+ if (ast_media_cache_init()) {
+ printf("Failed: ast_media_cache_init\n%s", term_quit());
+ exit(1);
+ }
+
/* loads the cli_permissoins.conf file needed to implement cli restrictions. */
ast_cli_perms_init(0);
diff --git a/main/bucket.c b/main/bucket.c
index f7845c8..7b8c689 100644
--- a/main/bucket.c
+++ b/main/bucket.c
@@ -284,7 +284,7 @@
if (ast_strlen_zero(name) || !bucket || !file ||
!bucket->create || !bucket->delete || !bucket->retrieve_id ||
- !create_cb) {
+ (!bucket->create && !create_cb)) {
return -1;
}
@@ -738,7 +738,7 @@
ast_string_field_set(file, scheme, uri_scheme);
- if (scheme->create(file)) {
+ if (scheme->create && scheme->create(file)) {
ao2_ref(file, -1);
return NULL;
}
diff --git a/main/media_cache.c b/main/media_cache.c
new file mode 100644
index 0000000..47173c8
--- /dev/null
+++ b/main/media_cache.c
@@ -0,0 +1,490 @@
+/*
+ * Asterisk -- An open source telephony toolkit.
+ *
+ * Copyright (C) 2015, Matt Jordan
+ *
+ * Matt Jordan <mjordan at digium.com>
+ *
+ * See http://www.asterisk.org for more information about
+ * the Asterisk project. Please do not directly contact
+ * any of the maintainers of this project for assistance;
+ * the project provides a web site, mailing lists and IRC
+ * channels for your use.
+ *
+ * This program is free software, distributed under the terms of
+ * the GNU General Public License Version 2. See the LICENSE file
+ * at the top of the source tree.
+ */
+
+/*!
+ * \file
+ * \brief An in-memory media cache
+ *
+ * \author \verbatim Matt Jordan <mjordan at digium.com> \endverbatim
+ *
+ */
+
+/*** MODULEINFO
+ <support_level>core</support_level>
+ ***/
+
+#include "asterisk.h"
+
+ASTERISK_REGISTER_FILE()
+
+#include <sys/stat.h>
+#include "asterisk/config.h"
+#include "asterisk/bucket.h"
+#include "asterisk/astdb.h"
+#include "asterisk/media_cache.h"
+
+/*! The name of the AstDB family holding items in the cache. */
+#define AST_DB_FAMILY "MediaCache"
+
+/*! Length of 'MediaCache' + 2 '/' characters */
+#define AST_DB_FAMILY_LEN 12
+
+/*! Number of buckets in the ao2 container holding our media items */
+#define AO2_BUCKETS 61
+
+/*! Our one and only container holding media items */
+static struct ao2_container *media_cache;
+
+/*!
+ * \internal
+ * \brief Hashing function for file metadata
+ */
+static int media_cache_hash(const void *obj, const int flags)
+{
+ const struct ast_bucket_file *object;
+ const char *key;
+
+ switch (flags & OBJ_SEARCH_MASK) {
+ case OBJ_SEARCH_KEY:
+ key = obj;
+ break;
+ case OBJ_SEARCH_OBJECT:
+ object = obj;
+ key = ast_sorcery_object_get_id(object);
+ break;
+ default:
+ /* Hash can only work on something with a full key */
+ ast_assert(0);
+ return 0;
+ }
+ return ast_str_hash(key);
+}
+
+/*!
+ * \internal
+ * \brief Comparison function for file metadata
+ */
+static int media_cache_cmp(void *obj, void *arg, int flags)
+{
+ struct ast_bucket_file *left = obj;
+ struct ast_bucket_file *right = arg;
+ const char *right_key = arg;
+ int cmp;
+
+ switch (flags & OBJ_SEARCH_MASK) {
+ case OBJ_SEARCH_OBJECT:
+ right_key = ast_sorcery_object_get_id(right);
+ /* Fall through */
+ case OBJ_SEARCH_KEY:
+ cmp = strcmp(ast_sorcery_object_get_id(left), right_key);
+ break;
+ case OBJ_SEARCH_PARTIAL_KEY:
+ cmp = strncmp(ast_sorcery_object_get_id(left), right_key, strlen(right_key));
+ break;
+ default:
+ ast_assert(0);
+ cmp = 0;
+ break;
+ }
+
+ return cmp ? 0 : CMP_MATCH | CMP_STOP;
+}
+
+
+int ast_media_cache_exists(const char *uri)
+{
+ struct ast_bucket_file *bucket_file;
+
+ if (ast_strlen_zero(uri)) {
+ return 0;
+ }
+
+ bucket_file = ao2_find(media_cache, uri, OBJ_SEARCH_KEY);
+ if (bucket_file) {
+ ao2_ref(bucket_file, -1);
+ return 1;
+ }
+
+ /* Check to see if any bucket implementation could return this item */
+ bucket_file = ast_bucket_file_retrieve(uri);
+ if (bucket_file) {
+ ao2_ref(bucket_file, -1);
+ return 1;
+ }
+
+ return 0;
+}
+
+/*!
+ * \internal
+ * \brief Sync \c bucket_file metadata to the AstDB
+ */
+static int metadata_sync_to_astdb(void *obj, void *arg, int flags)
+{
+ struct ast_bucket_metadata *metadata = obj;
+ const char *hash = arg;
+
+ ast_db_put(hash, metadata->name, metadata->value);
+
+ return 0;
+}
+
+/*!
+ * \internal
+ * \brief Sync a media cache item to the AstDB
+ * \param bucket_file The \c ast_bucket_file media cache item to sync
+ */
+static void media_cache_item_sync_to_astdb(struct ast_bucket_file *bucket_file)
+{
+ char hash[41]; /* 40 character SHA1 hash */
+
+ ast_sha1_hash(hash, ast_sorcery_object_get_id(bucket_file));
+ if (ast_db_put(AST_DB_FAMILY, ast_sorcery_object_get_id(bucket_file), hash)) {
+ return;
+ }
+
+ ast_db_put(hash, "path", bucket_file->path);
+ ast_bucket_file_metadata_callback(bucket_file, metadata_sync_to_astdb, hash);
+}
+
+/*!
+ * \internal
+ * \brief Delete a media cache item from the AstDB
+ * \param bucket_file The \c ast_bucket_file media cache item to delete
+ */
+static void media_cache_item_del_from_astdb(struct ast_bucket_file *bucket_file)
+{
+ char *hash_value;
+
+ if (ast_db_get_allocated(AST_DB_FAMILY, ast_sorcery_object_get_id(bucket_file), &hash_value)) {
+ return;
+ }
+
+ ast_db_deltree(hash_value, NULL);
+ ast_db_del(AST_DB_FAMILY, hash_value);
+ ast_free(hash_value);
+}
+
+/*!
+ * \internal
+ * \brief Update the name of the file backing a \c bucket_file
+ * \param preferred_file_name The preferred name of the backing file
+ */
+static void bucket_file_update_path(struct ast_bucket_file *bucket_file,
+ const char *preferred_file_name)
+{
+ if (ast_strlen_zero(preferred_file_name)) {
+ return;
+ }
+
+ if (!strcmp(bucket_file->path, preferred_file_name)) {
+ return;
+ }
+
+ rename(bucket_file->path, preferred_file_name);
+ ast_copy_string(bucket_file->path, preferred_file_name,
+ sizeof(bucket_file->path));
+}
+
+int ast_media_cache_retrieve(const char *uri, const char *preferred_file_name,
+ char *file_path, size_t len)
+{
+ struct ast_bucket_file *bucket_file;
+ SCOPED_AO2LOCK(media_lock, media_cache);
+
+ if (ast_strlen_zero(uri)) {
+ return -1;
+ }
+
+ /* First, retrieve from the ao2 cache here. If we find a bucket_file
+ * matching the requested URI, ask the appropriate backend if it is
+ * stale. If not; return it.
+ */
+ bucket_file = ao2_find(media_cache, uri, OBJ_SEARCH_KEY | OBJ_NOLOCK);
+ if (bucket_file) {
+ if (!ast_bucket_file_is_stale(bucket_file)) {
+ ast_copy_string(file_path, bucket_file->path, len);
+ ao2_ref(bucket_file, -1);
+ return 0;
+ }
+
+ /* Stale! Drop the ref, as we're going to retrieve it next. */
+ ao2_ref(bucket_file, -1);
+ }
+
+ /* Either this is new or the resource is stale; do a full retrieve
+ * from the appropriate bucket_file backend
+ */
+ bucket_file = ast_bucket_file_retrieve(uri);
+ if (!bucket_file) {
+ ast_log(LOG_WARNING, "Failed to obtain media at '%s'\n", uri);
+ return -1;
+ }
+
+ /* We can manipulate the 'immutable' bucket_file here, as we haven't
+ * let anyone know of its existence yet
+ */
+ bucket_file_update_path(bucket_file, preferred_file_name);
+ media_cache_item_sync_to_astdb(bucket_file);
+ ast_copy_string(file_path, bucket_file->path, len);
+ ao2_link_flags(media_cache, bucket_file, OBJ_NOLOCK);
+ ao2_ref(bucket_file, -1);
+
+ return 0;
+}
+
+int ast_media_cache_retrieve_metadata(const char *uri, const char *key,
+ char *value, size_t len)
+{
+ struct ast_bucket_file *bucket_file;
+ struct ast_bucket_metadata *metadata;
+
+ if (ast_strlen_zero(uri) || ast_strlen_zero(key) || !value) {
+ return -1;
+ }
+
+ bucket_file = ao2_find(media_cache, uri, OBJ_SEARCH_KEY);
+ if (!bucket_file) {
+ return -1;
+ }
+
+ metadata = ao2_find(bucket_file->metadata, key, OBJ_SEARCH_KEY);
+ if (!metadata) {
+ ao2_ref(bucket_file, -1);
+ return -1;
+ }
+ ast_copy_string(value, metadata->value, len);
+
+ ao2_ref(metadata, -1);
+ ao2_ref(bucket_file, -1);
+ return 0;
+}
+
+int ast_media_cache_create_or_update(const char *uri, const char *file_path,
+ struct ast_variable *metadata)
+{
+ struct ast_bucket_file *bucket_file;
+ struct ast_variable *it_metadata;
+ struct stat st;
+ char tmp[128];
+ char *ext;
+ char *file_path_ptr;
+ int created = 0;
+ SCOPED_AO2LOCK(media_lock, media_cache);
+
+ if (ast_strlen_zero(file_path) || ast_strlen_zero(uri)) {
+ return -1;
+ }
+ file_path_ptr = ast_strdupa(file_path);
+
+ if (stat(file_path, &st)) {
+ ast_log(LOG_WARNING, "Unable to obtain information for file %s for URI %s\n",
+ file_path, uri);
+ return -1;
+ }
+
+ bucket_file = ao2_find(media_cache, uri, OBJ_SEARCH_KEY | OBJ_NOLOCK);
+ if (bucket_file) {
+ struct ast_bucket_file *clone;
+
+ clone = ast_bucket_file_clone(bucket_file);
+ if (!clone) {
+ ao2_ref(bucket_file, -1);
+ return -1;
+ }
+
+ /* Remove the old bucket_file. We'll replace it if we succeed below. */
+ ao2_unlink_flags(media_cache, bucket_file, OBJ_NOLOCK);
+ ao2_ref(bucket_file, -1);
+
+ bucket_file = clone;
+ } else {
+ bucket_file = ast_bucket_file_alloc(uri);
+ if (!bucket_file) {
+ ast_log(LOG_WARNING, "Failed to create file storage for %s and %s\n",
+ uri, file_path);
+ return -1;
+ }
+ created = 1;
+ }
+
+ strcpy(bucket_file->path, file_path);
+ bucket_file->created.tv_sec = st.st_ctime;
+ bucket_file->modified.tv_sec = st.st_mtime;
+
+ snprintf(tmp, sizeof(tmp), "%ld", (long)st.st_atime);
+ ast_bucket_file_metadata_set(bucket_file, "accessed", tmp);
+
+ snprintf(tmp, sizeof(tmp), "%jd", (intmax_t)st.st_size);
+ ast_bucket_file_metadata_set(bucket_file, "size", tmp);
+
+ ext = strrchr(file_path_ptr, '.');
+ if (ext) {
+ ast_bucket_file_metadata_set(bucket_file, "ext", ext + 1);
+ }
+
+ for (it_metadata = metadata; it_metadata; it_metadata = it_metadata->next) {
+ ast_bucket_file_metadata_set(bucket_file, it_metadata->name, it_metadata->value);
+ }
+
+ if (created && ast_bucket_file_create(bucket_file)) {
+ ast_log(LOG_WARNING, "Failed to create media for %s\n", uri);
+ ao2_ref(bucket_file, -1);
+ return -1;
+ }
+ media_cache_item_sync_to_astdb(bucket_file);
+
+ ao2_link_flags(media_cache, bucket_file, OBJ_NOLOCK);
+ ao2_ref(bucket_file, -1);
+ return 0;
+}
+
+int ast_media_cache_delete(const char *uri)
+{
+ struct ast_bucket_file *bucket_file;
+ int res;
+
+ if (ast_strlen_zero(uri)) {
+ return -1;
+ }
+
+ bucket_file = ao2_find(media_cache, uri, OBJ_SEARCH_KEY | OBJ_UNLINK);
+ if (!bucket_file) {
+ return -1;
+ }
+
+ res = ast_bucket_file_delete(bucket_file);
+ media_cache_item_del_from_astdb(bucket_file);
+
+ ao2_ref(bucket_file, -1);
+
+ return res;
+}
+
+/*!
+ * \internal
+ * \brief Shutdown the media cache
+ */
+static void media_cache_shutdown(void)
+{
+ ao2_ref(media_cache, -1);
+ media_cache = NULL;
+}
+
+/*!
+ * \internal
+ * \brief Remove a media cache item from the AstDB
+ * \param uri The unique URI that represents the item in the cache
+ * \param hash The hash key for the item in the AstDB
+ */
+static void media_cache_remove_from_astdb(const char *uri, const char *hash)
+{
+ ast_db_del(AST_DB_FAMILY, uri + AST_DB_FAMILY_LEN);
+ ast_db_deltree(hash, NULL);
+}
+
+/*!
+ * \internal
+ * \brief Create an item in the media cache from entries in the AstDB
+ * \param uri The unique URI that represents the item in the cache
+ * \param hash The hash key for the item in the AstDB
+ * \retval 0 success
+ * \retval -1 failure
+ */
+static int media_cache_item_populate_from_astdb(const char *uri, const char *hash)
+{
+ struct ast_bucket_file *bucket_file;
+ struct ast_db_entry *db_tree;
+ struct ast_db_entry *db_entry;
+ struct stat st;
+
+ bucket_file = ast_bucket_file_alloc(uri);
+ if (!bucket_file) {
+ return -1;
+ }
+
+ db_tree = ast_db_gettree(hash, NULL);
+ for (db_entry = db_tree; db_entry; db_entry = db_entry->next) {
+ const char *key = strchr(db_entry->key + 1, '/');
+
+ if (ast_strlen_zero(key)) {
+ continue;
+ }
+ key++;
+
+ if (!strcasecmp(key, "path")) {
+ strcpy(bucket_file->path, db_entry->data);
+
+ if (stat(bucket_file->path, &st)) {
+ ast_log(LOG_WARNING, "Unable to obtain information for file %s for URI %s\n",
+ bucket_file->path, uri);
+ ao2_ref(bucket_file, -1);
+ ast_db_freetree(db_tree);
+ return -1;
+ }
+ } else {
+ ast_bucket_file_metadata_set(bucket_file, key, db_entry->data);
+ }
+ }
+ ast_db_freetree(db_tree);
+
+ if (ast_strlen_zero(bucket_file->path)) {
+ ao2_ref(bucket_file, -1);
+ ast_log(LOG_WARNING, "Failed to restore media cache item for '%s' from AstDB: no 'path' specified\n",
+ uri);
+ return -1;
+ }
+
+ ao2_link(media_cache, bucket_file);
+ ao2_ref(bucket_file, -1);
+
+ return 0;
+}
+
+/*!
+ * \internal
+ * \brief Populate the media cache from entries in the AstDB
+ */
+static void media_cache_populate_from_astdb(void)
+{
+ struct ast_db_entry *db_entry;
+ struct ast_db_entry *db_tree;
+
+ db_tree = ast_db_gettree(AST_DB_FAMILY, NULL);
+ for (db_entry = db_tree; db_entry; db_entry = db_entry->next) {
+ if (media_cache_item_populate_from_astdb(db_entry->key + AST_DB_FAMILY_LEN, db_entry->data)) {
+ media_cache_remove_from_astdb(db_entry->key, db_entry->data);
+ }
+ }
+ ast_db_freetree(db_tree);
+}
+
+int ast_media_cache_init(void)
+{
+ ast_register_atexit(media_cache_shutdown);
+
+ media_cache = ao2_container_alloc_options(AO2_ALLOC_OPT_LOCK_RWLOCK, AO2_BUCKETS,
+ media_cache_hash, media_cache_cmp);
+ if (!media_cache) {
+ return -1;
+ }
+
+ media_cache_populate_from_astdb();
+
+ return 0;
+}
diff --git a/tests/test_media_cache.c b/tests/test_media_cache.c
new file mode 100644
index 0000000..685693c
--- /dev/null
+++ b/tests/test_media_cache.c
@@ -0,0 +1,415 @@
+/*
+ * Asterisk -- An open source telephony toolkit.
+ *
+ * Copyright (C) 2015, Matt Jordan
+ *
+ * Matt Jordan <mjordan at digium.com>
+ *
+ * See http://www.asterisk.org for more information about
+ * the Asterisk project. Please do not directly contact
+ * any of the maintainers of this project for assistance;
+ * the project provides a web site, mailing lists and IRC
+ * channels for your use.
+ *
+ * This program is free software, distributed under the terms of
+ * the GNU General Public License Version 2. See the LICENSE file
+ * at the top of the source tree.
+ */
+
+/*!
+ * \file
+ * \brief Tests for the media cache API
+ *
+ * \author \verbatim Matt Jordan <mjordan at digium.com> \endverbatim
+ *
+ * \ingroup tests
+ */
+
+/*** MODULEINFO
+ <depend>TEST_FRAMEWORK</depend>
+ <support_level>core</support_level>
+ ***/
+
+#include "asterisk.h"
+
+ASTERISK_REGISTER_FILE()
+
+#include "asterisk/utils.h"
+#include "asterisk/module.h"
+#include "asterisk/test.h"
+#include "asterisk/bucket.h"
+#include "asterisk/media_cache.h"
+
+/*! The unit test category */
+#define CATEGORY "/main/media_cache/"
+
+/*! A 'valid' resource for the test bucket behind the media cache facade */
+#define VALID_RESOURCE "httptest://localhost:8088/test_media_cache/monkeys.wav"
+
+/*! An 'invalid' resource for the test bucket behind the media cache facade */
+#define INVALID_RESOURCE "httptest://localhost:8088/test_media_cache/bad.wav"
+
+/*! An 'invalid' scheme, not mapping to a valid bucket backend */
+#define INVALID_SCHEME "foo://localhost:8088/test_media_cache/monkeys.wav"
+
+/*! A URI with no scheme */
+#define NO_SCHEME "localhost:8088/test_media_cache/monkeys.wav"
+
+/*!
+ * \internal
+ * \brief Create callback for the httptest bucket backend
+ */
+static int bucket_http_test_wizard_create(const struct ast_sorcery *sorcery, void *data,
+ void *object)
+{
+ if (!strcmp(ast_sorcery_object_get_id(object), VALID_RESOURCE)) {
+ return 0;
+ }
+
+ return -1;
+}
+
+/*!
+ * \internal
+ * \brief Update callback for the httptest bucket backend
+ */
+static int bucket_http_test_wizard_update(const struct ast_sorcery *sorcery, void *data,
+ void *object)
+{
+ if (!strcmp(ast_sorcery_object_get_id(object), VALID_RESOURCE)) {
+ return 0;
+ }
+
+ return -1;
+}
+
+/*!
+ * \internal
+ * \brief Retrieve callback for the httptest bucket backend
+ */
+static void *bucket_http_test_wizard_retrieve_id(const struct ast_sorcery *sorcery,
+ void *data, const char *type, const char *id)
+{
+ struct ast_bucket_file *bucket_file;
+
+ if (!strcmp(type, "file") && !strcmp(id, VALID_RESOURCE)) {
+ bucket_file = ast_bucket_file_alloc(id);
+ if (!bucket_file) {
+ return NULL;
+ }
+
+ ast_bucket_file_temporary_create(bucket_file);
+ return bucket_file;
+ }
+ return NULL;
+}
+
+/*!
+ * \internal
+ * \brief Delete callback for the httptest bucket backend
+ */
+static int bucket_http_test_wizard_delete(const struct ast_sorcery *sorcery, void *data,
+ void *object)
+{
+ if (!strcmp(ast_sorcery_object_get_id(object), VALID_RESOURCE)) {
+ return 0;
+ }
+
+ return -1;
+}
+
+static struct ast_sorcery_wizard bucket_test_wizard = {
+ .name = "httptest",
+ .create = bucket_http_test_wizard_create,
+ .retrieve_id = bucket_http_test_wizard_retrieve_id,
+ .delete = bucket_http_test_wizard_delete,
+};
+
+static struct ast_sorcery_wizard bucket_file_test_wizard = {
+ .name = "httptest",
+ .create = bucket_http_test_wizard_create,
+ .update = bucket_http_test_wizard_update,
+ .retrieve_id = bucket_http_test_wizard_retrieve_id,
+ .delete = bucket_http_test_wizard_delete,
+};
+
+AST_TEST_DEFINE(exists_nominal)
+{
+ int res;
+
+ switch (cmd) {
+ case TEST_INIT:
+ info->name = __func__;
+ info->category = CATEGORY;
+ info->summary = "Test nominal existance of resources in the cache";
+ info->description =
+ "This test verifies that if a known resource is in the cache, "
+ "calling ast_media_cache_exists will return logical True. If "
+ "a resource does not exist, the same function call will return "
+ "logical False.";
+ return AST_TEST_NOT_RUN;
+ case TEST_EXECUTE:
+ break;
+ }
+
+ res = ast_media_cache_exists(INVALID_RESOURCE);
+ ast_test_validate(test, res == 0);
+
+ res = ast_media_cache_exists(VALID_RESOURCE);
+ ast_test_validate(test, res == 1);
+
+ return AST_TEST_PASS;
+}
+
+AST_TEST_DEFINE(exists_off_nominal)
+{
+ int res;
+
+ switch (cmd) {
+ case TEST_INIT:
+ info->name = __func__;
+ info->category = CATEGORY;
+ info->summary = "Test off nominal existance of resources in the cache";
+ info->description =
+ "This test verifies that checking for bad resources (NULL, bad "
+ "scheme, etc.) does not result in false positivies.";
+ return AST_TEST_NOT_RUN;
+ case TEST_EXECUTE:
+ break;
+ }
+
+ res = ast_media_cache_exists("");
+ ast_test_validate(test, res != 1);
+
+ res = ast_media_cache_exists(NULL);
+ ast_test_validate(test, res != 1);
+
+ res = ast_media_cache_exists(NO_SCHEME);
+ ast_test_validate(test, res != 1);
+
+ res = ast_media_cache_exists(INVALID_SCHEME);
+ ast_test_validate(test, res != 1);
+
+ return AST_TEST_PASS;
+}
+
+AST_TEST_DEFINE(create_update_nominal)
+{
+ int res;
+ char file_path[PATH_MAX];
+ char tmp_path_one[PATH_MAX] = "/tmp/test-media-cache-XXXXXX";
+ char tmp_path_two[PATH_MAX] = "/tmp/test-media-cache-XXXXXX";
+ int fd;
+
+ switch (cmd) {
+ case TEST_INIT:
+ info->name = __func__;
+ info->category = CATEGORY;
+ info->summary = "Test nominal creation/updating of a resource";
+ info->description =
+ "This test creates a resource and associates it with a file. "
+ "It then updates the resource with a new file. In both cases, "
+ "the test verifies that the resource is associated with the "
+ "file.";
+ return AST_TEST_NOT_RUN;
+ case TEST_EXECUTE:
+ break;
+ }
+
+ /* Create two local files to associate with a resource */
+ fd = mkstemp(tmp_path_one);
+ if (fd < 0) {
+ ast_test_status_update(test, "Failed to create first tmp file: %s\n",
+ tmp_path_one);
+ return AST_TEST_FAIL;
+ }
+ /* We don't need anything in the file */
+ close(fd);
+
+ fd = mkstemp(tmp_path_two);
+ if (fd < 0) {
+ ast_test_status_update(test, "Failed to create second tmp file: %s\n",
+ tmp_path_two);
+ return AST_TEST_FAIL;
+ }
+ close(fd);
+
+ ast_test_status_update(test, "Creating resource with %s\n", tmp_path_one);
+ res = ast_media_cache_create_or_update(VALID_RESOURCE, tmp_path_one, NULL);
+ ast_test_validate(test, res == 0);
+
+ res = ast_media_cache_retrieve(VALID_RESOURCE, NULL, file_path, PATH_MAX);
+ ast_test_status_update(test, "Got %s for first file path\n", file_path);
+ ast_test_validate(test, res == 0);
+ ast_test_validate(test, strcmp(file_path, tmp_path_one) == 0);
+
+ ast_test_status_update(test, "Creating resource with %s\n", tmp_path_two);
+ res = ast_media_cache_create_or_update(VALID_RESOURCE, tmp_path_two, NULL);
+ ast_test_validate(test, res == 0);
+
+ res = ast_media_cache_retrieve(VALID_RESOURCE, NULL, file_path, PATH_MAX);
+ ast_test_status_update(test, "Got %s for second file path\n", file_path);
+ ast_test_validate(test, res == 0);
+ ast_test_validate(test, strcmp(file_path, tmp_path_two) == 0);
+
+ unlink(tmp_path_one);
+ unlink(tmp_path_two);
+
+ return AST_TEST_PASS;
+}
+
+AST_TEST_DEFINE(create_update_off_nominal)
+{
+ int res;
+ char tmp_path[PATH_MAX] = "/tmp/test-media-cache-XXXXXX";
+ int fd;
+
+ switch (cmd) {
+ case TEST_INIT:
+ info->name = __func__;
+ info->category = CATEGORY;
+ info->summary = "Test off nominal creation/updating of a resource";
+ info->description =
+ "Test creation/updating of a resource with a variety of invalid\n"
+ "inputs.";
+ return AST_TEST_NOT_RUN;
+ case TEST_EXECUTE:
+ break;
+ }
+
+ /* Create two local files to associate with a resource */
+ fd = mkstemp(tmp_path);
+ if (fd < 0) {
+ ast_test_status_update(test, "Failed to create first tmp file: %s\n",
+ tmp_path);
+ return AST_TEST_FAIL;
+ }
+ /* We don't need anything in the file */
+ close(fd);
+
+ res = ast_media_cache_create_or_update(VALID_RESOURCE, NULL, NULL);
+ ast_test_validate(test, res != 0);
+
+ res = ast_media_cache_create_or_update(VALID_RESOURCE, "", NULL);
+ ast_test_validate(test, res != 0);
+
+ res = ast_media_cache_create_or_update(VALID_RESOURCE, "I don't exist", NULL);
+ ast_test_validate(test, res != 0);
+
+ res = ast_media_cache_create_or_update(INVALID_RESOURCE, tmp_path, NULL);
+ ast_test_validate(test, res != 0);
+
+ res = ast_media_cache_create_or_update(INVALID_SCHEME, tmp_path, NULL);
+ ast_test_validate(test, res != 0);
+
+ res = ast_media_cache_create_or_update(NO_SCHEME, tmp_path, NULL);
+ ast_test_validate(test, res != 0);
+
+ unlink(tmp_path);
+
+ return AST_TEST_PASS;
+}
+
+AST_TEST_DEFINE(create_update_metadata)
+{
+ int res;
+ char tmp_path[PATH_MAX] = "/tmp/test-media-cache-XXXXXX";
+ char file_path[PATH_MAX];
+ char actual_metadata[32];
+ struct ast_variable *meta_list = NULL;
+ struct ast_variable *tmp;
+ int fd;
+
+ switch (cmd) {
+ case TEST_INIT:
+ info->name = __func__;
+ info->category = CATEGORY;
+ info->summary = "Test nominal creation/updating of a resource";
+ info->description =
+ "This test creates a resource and associates it with a file. "
+ "It then updates the resource with a new file. In both cases, "
+ "the test verifies that the resource is associated with the "
+ "file.";
+ return AST_TEST_NOT_RUN;
+ case TEST_EXECUTE:
+ break;
+ }
+
+ /* Create two local files to associate with a resource */
+ fd = mkstemp(tmp_path);
+ if (fd < 0) {
+ ast_test_status_update(test, "Failed to create first tmp file: %s\n",
+ tmp_path);
+ return AST_TEST_FAIL;
+ }
+ /* We don't need anything in the file */
+ close(fd);
+
+ tmp = ast_variable_new("meta1", "value1", __FILE__);
+ if (!tmp) {
+ ast_test_status_update(test, "Failed to create metadata 1 for test\n");
+ return AST_TEST_FAIL;
+ }
+ ast_variable_list_append(&meta_list, tmp);
+
+ tmp = ast_variable_new("meta2", "value2", __FILE__);
+ if (!tmp) {
+ ast_test_status_update(test, "Failed to create metadata 2 for test\n");
+ return AST_TEST_FAIL;
+ }
+ ast_variable_list_append(&meta_list, tmp);
+
+ res = ast_media_cache_create_or_update(VALID_RESOURCE, tmp_path, meta_list);
+ ast_test_validate(test, res == 0);
+
+ res = ast_media_cache_retrieve(VALID_RESOURCE, NULL, file_path, PATH_MAX);
+ ast_test_status_update(test, "Got %s for second file path\n", file_path);
+ ast_test_validate(test, res == 0);
+ ast_test_validate(test, strcmp(file_path, tmp_path) == 0);
+
+ res = ast_media_cache_retrieve_metadata(VALID_RESOURCE, "meta1",
+ actual_metadata, sizeof(actual_metadata));
+ ast_test_validate(test, res == 0);
+ ast_test_validate(test, strcmp(actual_metadata, "value1") == 0);
+
+ res = ast_media_cache_retrieve_metadata(VALID_RESOURCE, "meta2",
+ actual_metadata, sizeof(actual_metadata));
+ ast_test_validate(test, res == 0);
+ ast_test_validate(test, strcmp(actual_metadata, "value2") == 0);
+
+ unlink(tmp_path);
+
+ return AST_TEST_PASS;
+}
+
+static int unload_module(void)
+{
+ AST_TEST_UNREGISTER(exists_nominal);
+ AST_TEST_UNREGISTER(exists_off_nominal);
+
+ AST_TEST_UNREGISTER(create_update_nominal);
+ AST_TEST_UNREGISTER(create_update_metadata);
+ AST_TEST_UNREGISTER(create_update_off_nominal);
+
+ return 0;
+}
+
+static int load_module(void)
+{
+ if (ast_bucket_scheme_register("httptest", &bucket_test_wizard,
+ &bucket_file_test_wizard, NULL, NULL)) {
+ ast_log(LOG_ERROR, "Failed to register Bucket HTTP test wizard scheme implementation\n");
+ return AST_MODULE_LOAD_FAILURE;
+ }
+
+ AST_TEST_REGISTER(exists_nominal);
+ AST_TEST_REGISTER(exists_off_nominal);
+
+ AST_TEST_REGISTER(create_update_nominal);
+ AST_TEST_REGISTER(create_update_metadata);
+ AST_TEST_REGISTER(create_update_off_nominal);
+
+ return AST_MODULE_LOAD_SUCCESS;
+}
+
+AST_MODULE_INFO_STANDARD(ASTERISK_GPL_KEY, "Media Cache Tests");
--
To view, visit https://gerrit.asterisk.org/487
To unsubscribe, visit https://gerrit.asterisk.org/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I11227abbf14d8929eeb140ddd101dd5c3820391e
Gerrit-PatchSet: 6
Gerrit-Project: asterisk
Gerrit-Branch: master
Gerrit-Owner: Matt Jordan <mjordan at digium.com>
Gerrit-Reviewer: Anonymous Coward #1000019
Gerrit-Reviewer: Joshua Colp <jcolp at digium.com>
Gerrit-Reviewer: Matt Jordan <mjordan at digium.com>
More information about the asterisk-commits
mailing list