[asterisk-commits] oej: branch oej/appleraisin-astdb-realtime-1.8 r317672 - in /team/oej/applera...
SVN commits to the Asterisk project
asterisk-commits at lists.digium.com
Fri May 6 11:37:05 CDT 2011
Author: oej
Date: Fri May 6 11:37:01 2011
New Revision: 317672
URL: http://svnview.digium.com/svn/asterisk?view=rev&rev=317672
Log:
Adding the patch for the 1.8 branch for testing
Added:
team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin (with props)
Modified:
team/oej/appleraisin-astdb-realtime-1.8/configs/extconfig.conf.sample
team/oej/appleraisin-astdb-realtime-1.8/main/db.c
Added: team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin
URL: http://svnview.digium.com/svn/asterisk/team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin?view=auto&rev=317672
==============================================================================
--- team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin (added)
+++ team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin Fri May 6 11:37:01 2011
@@ -1,0 +1,58 @@
+Edvina AB
+Olle E. Johansson March 2009
+
+
+
+AppleRaisin - Moving Astdb to realtime
+--------------------------------------
+
+This branch, based on Asterisk svn trunk, implements storage of AstDB in ARA, the realtime data object
+manipulation abstraction something in Asterisk. The name "realtime" is getting more and more
+non-descriptive for what ARA has become. With the addition of delete and store operations,
+realtime fits very well for astdb.
+
+Why?
+----
+
+There are a lot of applications and GUI's out there that makes heavy use of astdb. In order
+to be able to have some sort of failover solution and provide scalability, astdb needs to
+move to another engine than Berkeley DB v1 that uses one single file. The GUI I had in
+mind when working with this code was FreePBX from freepbx.org.
+
+Howto:
+------
+
+1. This branch requires realtime drivers with the store and delete methods implemented.
+ The regular asterisk-addons and Asterisk realtime drivers now implement this (in trunk).
+ I don't know the state of other than odbc and mysql drivers in this regard.
+
+2. Make sure you load the realtime driver before channel drivers and modules that use AstDb
+
+3. Note that because astdb is part of the Asterisk core, we have a chicken-and-egg issue. The AstDB
+ interface is initialized before any modules, so by default, astdb initializes to the
+ Berkeley DB interface. For each call, astdb checks if there's a realtime family activated
+ and switches as soon as this is available. If there's any code in-core using astdb
+ in your Asterisk version this will be an issue. I haven't identified any such issues
+ in the Digium distribution of Asterisk.
+
+4. Create a database with this table or something similar for non-SQL realtime drivers.
+ Configure this in the "astdb" family in extconfig.conf
+
+ CREATE TABLE `astdb` (
+ `systemname` varchar(40) NOT NULL,
+ `family` varchar(256) NOT NULL,
+ `keyname` varchar(256) NOT NULL,
+ `value` varchar(1050) NOT NULL,
+ PRIMARY KEY (`family`,`keyname`)
+ ) ENGINE=MyISAM DEFAULT CHARSET=latin1;
+
+5. Remember to set the systemname in asterisk.conf
+
+6. If you want to access an astdb variable in another server, just query using the normal
+ realtime or odbc functions.
+
+Thank you to...
+----------------
+- Jared Smith, who gently forced me into the #freepbx-dev IRC channel to discuss this project
+- During the process I've gotten a lot of information from Philippe @freepbx - thank you!
+- Micke Carlsson, who have tested the code for the FreePBX project
Propchange: team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin
------------------------------------------------------------------------------
svn:keywords = Author Date Id Revision
Propchange: team/oej/appleraisin-astdb-realtime-1.8/README.appleraisin
------------------------------------------------------------------------------
svn:mime-type = text/plain
Modified: team/oej/appleraisin-astdb-realtime-1.8/configs/extconfig.conf.sample
URL: http://svnview.digium.com/svn/asterisk/team/oej/appleraisin-astdb-realtime-1.8/configs/extconfig.conf.sample?view=diff&rev=317672&r1=317671&r2=317672
==============================================================================
--- team/oej/appleraisin-astdb-realtime-1.8/configs/extconfig.conf.sample (original)
+++ team/oej/appleraisin-astdb-realtime-1.8/configs/extconfig.conf.sample Fri May 6 11:37:01 2011
@@ -73,6 +73,7 @@
;queue_members => odbc,asterisk
;musiconhold => mysql,general
;queue_log => mysql,general
+;astdb => mysql,astdb
;
;
; While most dynamic realtime engines are automatically used when defined in
Modified: team/oej/appleraisin-astdb-realtime-1.8/main/db.c
URL: http://svnview.digium.com/svn/asterisk/team/oej/appleraisin-astdb-realtime-1.8/main/db.c?view=diff&rev=317672&r1=317671&r2=317672
==============================================================================
--- team/oej/appleraisin-astdb-realtime-1.8/main/db.c (original)
+++ team/oej/appleraisin-astdb-realtime-1.8/main/db.c Fri May 6 11:37:01 2011
@@ -25,6 +25,13 @@
* \note DB3 is licensed under Sleepycat Public License and is thus incompatible
* with GPL. To avoid having to make another exception (and complicate
* licensing even further) we elect to use DB1 which is BSD licensed
+ *
+ * \note AstDB realtime
+ * AstDB realtime works with the basic operations - put, get, del
+ * Database show also works.
+ *
+ * The tree/family operations doesn't currently work. Maybe tree and family needs
+ * to be separate fields in the database, instead of one single field as I've tried with.
*/
#include "asterisk.h"
@@ -46,6 +53,7 @@
#include "asterisk/utils.h"
#include "asterisk/lock.h"
#include "asterisk/manager.h"
+#include "asterisk/config.h"
#include "db1-ast/include/db.h"
/*** DOCUMENTATION
@@ -103,18 +111,109 @@
static DB *astdb;
AST_MUTEX_DEFINE_STATIC(dblock);
static ast_cond_t dbcond;
-
static void db_sync(void);
+static int db_rt; /*!< Flag for realtime system */
+static char *db_rt_rtfamily = "astdb"; /*!< Realtime name tag */
+static char *db_rt_value = "value"; /*!< Database field name for values */
+static char *db_rt_family = "family"; /*!< Database field name for family */
+static char *db_rt_key = "keyname"; /*!< Database field name for key */
+static char *db_rt_sysnamelabel = "systemname"; /*!< Database field name for system name */
+static const char *db_rt_sysname; /*!< From asterisk.conf or "asterisk" */
+
+/*! \brief Initialize either realtime support or Asterisk ast-db.
+
+ Note: Since realtime support is loaded after astdb, we can not do this early, but has to do the
+ check on need. Now, there's a risk an internal module (not a loaded module) use astdb before
+ realtime is checked, but that's something we propably have to live with until we solve it.
+
+ Make sure that realtime modules are loaded before dundi and the channels.
+*/
static int dbinit(void)
{
- if (!astdb && !(astdb = dbopen(ast_config_AST_DB, O_CREAT | O_RDWR, AST_FILE_MODE, DB_BTREE, NULL))) {
+ if (db_rt) {
+ return 0;
+ }
+ db_rt = ast_check_realtime(db_rt_rtfamily);
+ if (!db_rt && !astdb && !(astdb = dbopen(ast_config_AST_DB, O_CREAT | O_RDWR, AST_FILE_MODE, DB_BTREE, NULL))) {
ast_log(LOG_WARNING, "Unable to open Asterisk database '%s': %s\n", ast_config_AST_DB, strerror(errno));
return -1;
}
return 0;
}
+/*! \brief Load a set of entries from astdb/realtime. This is for all operations that
+ work on a whole "tree" or "family"
+ \note the calling function needs to destroy the result set with ast_config_destroy(resultset)
+*/
+static struct ast_variable *db_realtime_getall(const char *family, const char *key)
+{
+ struct ast_variable *data, *returnset = NULL;
+ const char *keyname = NULL, *familyname = NULL;
+ struct ast_config *variablelist = NULL;
+ const char *cat = NULL;
+ char buf[512];
+
+ ast_debug(2, ">>>>>> getall family: %s Key %s \n", family, key);
+
+ if (ast_strlen_zero(family)) {
+ /* Load all entries in the astdb */
+ if (ast_strlen_zero(key)) {
+ /* No variables given */
+ variablelist = ast_load_realtime_multientry(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, SENTINEL);
+ } else {
+ /* Only key given */
+ variablelist = ast_load_realtime_multientry(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_key, key, SENTINEL);
+ }
+ } else {
+ if (ast_strlen_zero(key)) {
+ variablelist = ast_load_realtime_multientry(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_family, family, SENTINEL);
+ } else {
+ variablelist = ast_load_realtime_multientry(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_family, family, db_rt_key, key, SENTINEL);
+ }
+ }
+ if (!variablelist) {
+ return NULL;
+ }
+ /* Now we need to start converting all this stuff. We have thre ast_variable sets per record in the result set */
+ while ((cat = ast_category_browse(variablelist, cat))) {
+ struct ast_variable *resultset, *cur;
+
+ cur = resultset = ast_variable_browse(variablelist, cat);
+
+ /* skip the system name */
+ while (cur) {
+ ast_debug(2, ">>>> Found name %s ...\n", cur->name);
+ if (!strcmp(cur->name, db_rt_family)) {
+ familyname = cur->value;
+ } else if (!strcmp(cur->name, db_rt_key)) {
+ keyname = cur->value;
+ } else if (!strcmp(cur->name, db_rt_value)) {
+ snprintf(buf, sizeof(buf), "/%s/%s", S_OR(familyname, ""), S_OR(keyname, ""));
+ data = ast_variable_new(buf, S_OR(cur->value, "astdb-realtime"), "");
+ familyname = keyname = NULL;
+ ast_debug(2, "#### Found Variable %s with value %s \n", buf, cur->value);
+ /* Add this to the returnset */
+ data->next = returnset;
+ returnset = data;
+ } else {
+ if (ast_strlen_zero(cur->name)) {
+ ast_debug(2, "#### Skipping strange record \n");
+ } else {
+ ast_debug(2, "#### Skipping %s with value %s \n", cur->name, cur->value);
+ }
+ }
+ cur = cur->next;
+ }
+ //if (resultset)
+ //ast_variables_destroy(resultset);
+ }
+
+ /* Clean up the resultset */
+ ast_config_destroy(variablelist);
+
+ return returnset;
+}
static inline int keymatch(const char *key, const char *prefix)
{
@@ -164,10 +263,34 @@
prefix[0] = '\0';
}
- ast_mutex_lock(&dblock);
- if (dbinit()) {
- ast_mutex_unlock(&dblock);
- return -1;
+ if (!db_rt) {
+ ast_mutex_lock(&dblock);
+ if (dbinit()) {
+ ast_mutex_unlock(&dblock);
+ return -1;
+ }
+ if (db_rt) {
+ ast_mutex_unlock(&dblock);
+ }
+ }
+ if (db_rt) {
+ struct ast_variable *murderlist, *cur;
+ cur = murderlist = db_realtime_getall(family, S_OR(keytree, ""));
+ while (cur) {
+ int res;
+ char *familyname = ast_strdupa(&cur->name[1]); /* Skip the first slash */
+ char *keyname = familyname;
+ familyname = strsep(&keyname, "/");
+
+ res = ast_destroy_realtime(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_family, familyname, db_rt_key, keyname, SENTINEL);
+ if (res >= 0)
+ counter ++;
+ cur = cur->next;
+ }
+
+ ast_variables_destroy(murderlist);
+
+ return counter;
}
memset(&key, 0, sizeof(key));
@@ -192,26 +315,51 @@
int ast_db_put(const char *family, const char *keys, const char *value)
{
- char fullkey[256];
DBT key, data;
- int res, fullkeylen;
-
- ast_mutex_lock(&dblock);
- if (dbinit()) {
+ int res;
+
+ if (!db_rt) {
+ ast_mutex_lock(&dblock);
+ if (dbinit()) {
+ ast_mutex_unlock(&dblock);
+ return -1;
+ }
+ if (db_rt)
+ ast_mutex_unlock(&dblock);
+ }
+
+ if (db_rt) {
+ int rowsaffected ;
+
+ /* Now, the question here is if we're overwriting or adding
+ First, let's try updating it.
+ */
+ ast_debug(2, ".... Trying ast_update_realtime\n");
+ /* Update_realtime with mysql returns the number of rows affected */
+ rowsaffected = ast_update2_realtime(db_rt_rtfamily, db_rt_family, family, db_rt_key, keys, db_rt_sysnamelabel, db_rt_sysname, SENTINEL, db_rt_value, value, SENTINEL);
+ res = rowsaffected > 0 ? 0 : 1;
+ if (res) {
+ ast_debug(2, ".... Trying ast_store_realtime\n");
+ /* Update failed, let's try adding a new record */
+ res = ast_store_realtime(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_family, family, db_rt_key, keys, db_rt_value, value, SENTINEL);
+ /* Ast_store_realtime with mysql returns 0 if ok, -1 if bad */
+
+ }
+ } else {
+ int fullkeylen;
+ char fullkey[256];
+
+ fullkeylen = snprintf(fullkey, sizeof(fullkey), "/%s/%s", family, keys);
+ memset(&key, 0, sizeof(key));
+ memset(&data, 0, sizeof(data));
+ key.data = fullkey;
+ key.size = fullkeylen + 1;
+ data.data = (char *) value;
+ data.size = strlen(value) + 1;
+ res = astdb->put(astdb, &key, &data, 0);
+ db_sync();
ast_mutex_unlock(&dblock);
- return -1;
- }
-
- fullkeylen = snprintf(fullkey, sizeof(fullkey), "/%s/%s", family, keys);
- memset(&key, 0, sizeof(key));
- memset(&data, 0, sizeof(data));
- key.data = fullkey;
- key.size = fullkeylen + 1;
- data.data = (char *) value;
- data.size = strlen(value) + 1;
- res = astdb->put(astdb, &key, &data, 0);
- db_sync();
- ast_mutex_unlock(&dblock);
+ }
if (res)
ast_log(LOG_WARNING, "Unable to put value '%s' for key '%s' in family '%s'\n", value, keys, family);
return res;
@@ -223,12 +371,38 @@
DBT key, data;
int res, fullkeylen;
- ast_mutex_lock(&dblock);
- if (dbinit()) {
- ast_mutex_unlock(&dblock);
- return -1;
- }
-
+ if (!db_rt) {
+ ast_mutex_lock(&dblock);
+ if (dbinit()) {
+ ast_mutex_unlock(&dblock);
+ return -1;
+ }
+ if (db_rt) {
+ ast_mutex_unlock(&dblock);
+ }
+ }
+
+ if (db_rt) {
+ struct ast_variable *var, *res;
+ memset(value, 0, valuelen);
+
+ res = var = ast_load_realtime(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_family, family, db_rt_key, keys, SENTINEL);
+ if (!var) {
+ return 1;
+ }
+ /* We should only have one value here, so let's make this simple... */
+ while (res) {
+ if (!strcasecmp(res->name, db_rt_value)) {
+ ast_copy_string(value, res->value, (valuelen > strlen(res->value) ) ? strlen(res->value) +1: valuelen);
+ res = NULL;
+ } else {
+ res = res->next;
+ }
+ }
+
+ ast_variables_destroy(var);
+ return 0;
+ }
fullkeylen = snprintf(fullkey, sizeof(fullkey), "/%s/%s", family, keys);
memset(&key, 0, sizeof(key));
memset(&data, 0, sizeof(data));
@@ -267,21 +441,29 @@
DBT key;
int res, fullkeylen;
- ast_mutex_lock(&dblock);
- if (dbinit()) {
+ if (!db_rt) {
+ ast_mutex_lock(&dblock);
+ if (dbinit()) {
+ ast_mutex_unlock(&dblock);
+ return -1;
+ }
+ if (db_rt)
+ ast_mutex_unlock(&dblock);
+ }
+
+ if (db_rt) {
+ int rowcount = ast_destroy_realtime(db_rt_rtfamily, db_rt_sysnamelabel, db_rt_sysname, db_rt_family, family, db_rt_key, keys, SENTINEL);
+ res = rowcount > 0 ? 0 : 1;
+ } else {
+ fullkeylen = snprintf(fullkey, sizeof(fullkey), "/%s/%s", family, keys);
+ memset(&key, 0, sizeof(key));
+ key.data = fullkey;
+ key.size = fullkeylen + 1;
+
+ res = astdb->del(astdb, &key, 0);
+ db_sync();
ast_mutex_unlock(&dblock);
- return -1;
- }
-
- fullkeylen = snprintf(fullkey, sizeof(fullkey), "/%s/%s", family, keys);
- memset(&key, 0, sizeof(key));
- key.data = fullkey;
- key.size = fullkeylen + 1;
-
- res = astdb->del(astdb, &key, 0);
- db_sync();
-
- ast_mutex_unlock(&dblock);
+ }
if (res) {
ast_debug(1, "Unable to find key '%s' in family '%s'\n", keys, family);
@@ -307,6 +489,7 @@
if (a->argc != 5)
return CLI_SHOWUSAGE;
+
res = ast_db_put(a->argv[2], a->argv[3], a->argv[4]);
if (res) {
ast_cli(a->fd, "Failed to update entry\n");
@@ -401,6 +584,29 @@
}
return CLI_SUCCESS;
}
+
+static void handle_cli_database_show_realtime(struct ast_cli_args *a, const char *family, const char *key)
+{
+ struct ast_variable *resultset;
+ struct ast_variable *cur;
+ int counter = 0;
+
+ dbinit();
+ if (!db_rt) {
+ ast_cli(a->fd, "Error: Can't connect to astdb/realtime\n");
+ return;
+ }
+
+ cur = resultset = db_realtime_getall(family, key);
+ while (cur) {
+ ast_cli(a->fd, "%-40s: %-25s\n", cur->name, S_OR(cur->value, ""));
+ cur = cur->next;
+ counter++;
+ }
+ ast_cli(a->fd, "%d results found.\n", counter);
+ ast_variables_destroy(resultset);
+}
+
static char *handle_cli_database_show(struct ast_cli_entry *e, int cmd, struct ast_cli_args *a)
{
@@ -435,6 +641,10 @@
} else {
return CLI_SHOWUSAGE;
}
+ if (db_rt) {
+ handle_cli_database_show_realtime(a, a->argc >= 3 ? a->argv[2] : "", a->argc == 4 ? a->argv[3] : "");
+ return CLI_SUCCESS;
+ }
ast_mutex_lock(&dblock);
if (dbinit()) {
ast_mutex_unlock(&dblock);
@@ -499,6 +709,13 @@
ast_cli(a->fd, "Database unavailable\n");
return CLI_SUCCESS;
}
+
+ if (db_rt) {
+ ast_mutex_unlock(&dblock);
+ handle_cli_database_show_realtime(a, "", a->argv[2]);
+ return CLI_SUCCESS;
+ }
+
memset(&key, 0, sizeof(key));
memset(&data, 0, sizeof(data));
pass = 0;
@@ -758,6 +975,12 @@
int astdb_init(void)
{
+ /* When this routine is run, the realtime modules are not loaded so we can't initialize realtime yet. */
+ db_rt = 0;
+
+ /* If you have multiple systems using the same database, set the systemname in asterisk.conf */
+ db_rt_sysname = S_OR(ast_config_AST_SYSTEM_NAME, "asterisk");
+
pthread_t dont_care;
ast_cond_init(&dbcond, NULL);
@@ -765,7 +988,9 @@
return -1;
}
+ /* initialize astdb or realtime */
dbinit();
+
ast_cli_register_multiple(cli_database, ARRAY_LEN(cli_database));
ast_manager_register_xml("DBGet", EVENT_FLAG_SYSTEM | EVENT_FLAG_REPORTING, manager_dbget);
ast_manager_register_xml("DBPut", EVENT_FLAG_SYSTEM, manager_dbput);
More information about the asterisk-commits
mailing list