[asterisk-commits] seanbright: branch group/NoLossCDR-Redux2 r105554 - /team/group/NoLossCDR-Red...
SVN commits to the Asterisk project
asterisk-commits at lists.digium.com
Sun Mar 2 17:03:23 CST 2008
Author: seanbright
Date: Sun Mar 2 17:03:22 2008
New Revision: 105554
URL: http://svn.digium.com/view/asterisk?view=rev&rev=105554
Log:
Do an intermediate check in of some documentation. This is a very rough cut, and there is a lot more to do, so cut me some slack :)
Juggie - Please look over what I've done and fix any mistakes you find (I'm sure there are some). I've also left your notes at the bottom which I will be integrating shortly.
Added:
team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt (with props)
Added: team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt
URL: http://svn.digium.com/view/asterisk/team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt?view=auto&rev=105554
==============================================================================
--- team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt (added)
+++ team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt Sun Mar 2 17:03:22 2008
@@ -1,0 +1,151 @@
+Lossless CDR
+============
+
+Lossless CDR (also referred to as NoLossCDR) is an attempt to overcome some of
+the shortcomings of the existing CDR infrastructure in Asterisk.
+
+Currently, Asterisk has two modes of posting CDRs:
+
+ 1) Immediate Mode
+ This is the mode that most installations currently use. A CDR (or series
+ of CDRs) are posted to the CDR subsystem and a given backend (described
+ below) writes the CDR record to the underlying storage medium - be it a
+ CSV file, the Asterisk Manager Interface (AMI) or a database connection.
+
+ One of the disadvantages of this approach is that when a channel posts
+ the CDR, the channel is kept alive while the record is being written.
+ For a heavily loaded server and for servers competing for the network,
+ this can be a time consuming and resource intensive process. Its
+ advantage, of course, is that records are posted (or at least there is an
+ attempt to post) in realtime.
+
+ 2) Batch Mode
+ In this mode, each record is stored in a queue for storage later. The
+ maximum size of this queue, and the duration between posts is
+ configurable by the server admin. There are 3 situations that cause the
+ CDRs to be written. Either the queue backlog reaches its maximum size,
+ the time between batches is exceeded, or the PBX admin manually causes
+ the records to be posted by running a CLI command.
+
+ If blocking is the greatest weakness of the immediate mode approach, this
+ is where batch mode has the upper hand. When batches are written, they
+ are written from a secondary thread that is not associated with the
+ channel posting the CDR. This allows channels to be built and torn down
+ more quickly, resulting in better performance overall.
+
+ That being said, its greatest weakness is that were Asterisk to shutdown
+ unsafely (power outage, segfault, etc.) all the records that had not been
+ posted would be lost.
+
+Lossless CDR attempts to address these concerns by off loading all CDR logging
+to separate threads. While initially this appears similar to batch mode (and
+some aspects of it are carried over) the difference is CDRs will be posted for
+logging immediately, while a secondary thread waits for new records to log.
+
+Backends
+--------
+
+A CDR Backend is an Asterisk module that implements logging of CDRs to a storage
+medium. Each Backend is responsible for reading its configuration file,
+registering itself with the CDR subsystem, and writing any ast_cdr records
+passed to it based on its implementation (flat-file, database, etc.).
+
+Currently, if a given Backend wished to allow multiple storage locations, it had
+to handle this internally. A great example of this is cdr_adaptive_odbc, which
+allows the server administrator to create multiple database connections and have
+them all logged to each time a single ast_cdr was posted to it.
+
+We have decided to pull this implementation up to the CDR system itself, rather
+than requiring each module to implement their own method. We do this by
+establishing a distinction between a CDR Backend and a Backend Sink. Under
+Lossless CDR, each Backend will support multiple output sinks, with minimal code
+changes to the current logging functions.
+
+Sinks
+-----
+
+For our purposes, a sink is simply a location where CDR records are ultimately
+stored. Internally, the CDR system will maintain a list of Backends and their
+associated output sinks, freeing the CDR module author from doing the list
+management themselves. After registering the Backend (by name, currently), the
+CDR module will simple register one or more sinks, using the Backend name as the
+key. Unregistering the Backend will cause all of the sinks to be unregistered
+as well.
+
+(TODO: Update struct example once backends and sinks are split)
+
+struct ast_cdr_backend {
+ AST_DECLARE_STRING_FIELDS(
+ AST_STRING_FIELD(name);
+ AST_STRING_FIELD(sink_name);
+ AST_STRING_FIELD(description);
+ );
+ ast_cdrbe be;
+ ast_cdrbe_cleanup cleanup;
+ void *config;
+ int cancel_thread;
+ int waiting_cdrs;
+ pthread_t cdr_thread;
+ ast_cond_t cdr_pending_cond;
+ ast_cond_t cdr_retry_cond;
+ ast_mutex_t retry_poll_lock;
+ AST_LIST_HEAD(, ast_cdr) cdr_queue;
+ AST_RWLIST_ENTRY(ast_cdr_backend) list;
+};
+
+Batch Mode has been removed because it serves no purpose in the noloss model.
+NoLoss will be inherantally non blocking and batchmode has not been shown to
+provide any performance gains.
+
+To implement NoLoss we have a struct called ast_cdr_beitem. It is as follows:
+
+struct ast_cdr_beitem {
+ char name[20];
+ char name_detail[20];
+ char desc[80];
+ ast_cdrbe be;
+ ast_cdrbe_cleanup cleanup;
+ void *be_data;
+ int cancel_thread;
+ int waiting_cdrs;
+ pthread_t cdr_thread;
+ ast_cond_t cdr_pending_cond;
+ ast_cond_t cdr_retry_cond;
+ ast_mutex_t retry_poll_lock;
+ AST_LIST_HEAD(, ast_cdr) cdr_queue;
+ AST_RWLIST_ENTRY(ast_cdr_beitem) list;
+};
+
+Each ast_cdr_beitem is a cdr backend registration, it contains among the usual
+information:
+
+be = The Function to call to post to.
+cleanup = The function to call on a shutdown.
+be_data = Something to distinguish between registrations in an engine with
+ multiple registrations.
+cancel_thread = A way to signal the thread to exit
+waiting_cdrs = How many cdrs this backend is waiting to post
+cdr_thread = The thread handle for the posting engine for this backend.
+
+cdr_pending_cond =
+cdr_retry_cond =
+retry_poll_lock =
++ the usual AST_LIST stuff.
+
+Model:
+
+* CDR Backend loads
+
+* CDR Backend calls ast_cdr_register as many times as it has configured end
+ points to write into
+
+* Each call if it is valid, generates an entry into ast_cdr_beitem, which in turn
+ generates a thread to manage the queue.
+
+* When a CDR is posted to the CDR engine, we loop all the registered backends,
+ dropping the cdr into each queue, incrementing the use count of the cdr, then
+ signaling the thread to wake up if it is infact asleep. Each thread will then
+ work to post the cdr for the queue it is responsible for.
+
+* As each backend posts and returns the use count is decremented, until we know
+ the cdr record is no longer in use and we call ast_cdr_free on the record.
Propchange: team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt
------------------------------------------------------------------------------
svn:keywords = Author Date Id Revision
Propchange: team/group/NoLossCDR-Redux2/doc/lossless_cdr.txt
------------------------------------------------------------------------------
svn:mime-type = text/plain
More information about the asterisk-commits
mailing list