[asterisk-dev] Violent Objections? CDR dst/dcontext probs in 1.4+ when a macro is involved.

[Steve Murphy]

On Tue, 2007-07-03 at 08:12 +0300, Tzafrir Cohen wrote:
> On Mon, Jul 02, 2007 at 10:41:46PM -0600, Steve Murphy wrote:
> > On Fri, 2007-06-29 at 07:30 -0500, Tilghman Lesher wrote:
> > > On Friday 29 June 2007, Benny Amorsen wrote:
> > > > >>>>> "TL" == Tilghman Lesher <tilghman at mail.jeffandtilghman.com> writes:
> > > >
> > > > TL> Nothing is preventing you from creating another CDR variable (and
> > > > TL> logging it) called Destination.
> > > >
> > > > You cannot tell the database backend to include special fields in
> > > > CDR's. Very inconvenient.
> > > >
> > > > I have a similar problem in this setup:
> > > >
> > > > [outgoing-sipprovider]
> > > >  exten => _X,1,Set(CALLERID(num)=00${CALLERID(num)})
> > > >  exten => _X,n,Dial(00${EXTEN}@sipprovider
> > > >
> > > > I want to log everything in E.264-format -- a US call would look like
> > > > this: source 11235556666, destination 11235557777. The sip provider
> > > > wants it like this: source 0011235556666, 0011235555777. If I could
> > > > manipulate the CDR variables directly, it would be relatively easy to
> > > > fix.
> > > 
> > > http://svn.digium.com/view/asterisk/trunk/cdr/cdr_adaptive_odbc.c
> > > 
> > > This module allows you to use custom fields, now.  It is compatible with
> > > 1.4, and if trunk is changed such that it is no longer compatible with 1.4,
> > > I will create a branch in svncommunity that will house cdr_adaptive_odbc
> > > for 1.4 (as a backport).  Due to architectural limitations, this module cannot
> > > be backported to 1.2.
> > > 
> > 
> > Tilghman made some excellent points. One of the major ones is that EVEN
> > IF I
> > ALLOW users to CHANGE CDR fields from the dialplan, it is hard to
> > predict what events might override the user's settings. Asterisk has a
> > ton of places that 
> > tweak CDR's, and in interesting ways.  So, even with the restrictions
> > removed
> > from the CDR() func, you still might end up frustrated and confused. 
> > 
> > His suggestion to use customizable backends like cdr_adaptive_odbc is an
> > excellent suggestion. It gives the user complete "final" control, no
> > experiments needed, no goto (or other non-obvious) hacks to get the
> > fields the way you want them. Correct me if I'm wrong, but it looks like
> > you could pack data into a
> > seperate column, and then replace any CDR field with the contents of
> > that column.
> 
> But then you would need to duplicate every backend.
> 
> Then again, what was the real point in the duplication of cdr_csv and
> cdr_custom ? I think people usually ended up using cdr_custom.
> 
> What is the point for this code duplication?
> 

A good question. For example, when cdr_custom was introduced, we could
have marked cdr_csv as deprecated, and removed it, say in 1.4 or
something. But we didn't. But we still can. I could see having both for
a time, as those using
cdr_csv need not change anything; If/when it disappears, they'll have to
define 
a cdr_custom, and the dir in which they find their CDR records will
change.

It's more a matter of actively defining the migration path, than of
active duplication. We've just chosen the 'easy' path for now.

We could easily upgrade all the existing back ends (where 'easily' is a
matter of relativity), and mark all the existing backends as deprecated,
and remove them in a release or two.

I believe that even cdr_custom will have to be upgraded.... 

For those who haven't peeked at the capabilities of adaptive_odbc,
here's a snippet from the config file:

  ...
  ; The columns in the tables should match up word-for-word
(case-insensitive)
  ; to the CDR variables set in the dialplan.  The natural advantage to
this
  ; system is that beyond setting up the configuration file to tell you
what
  ; tables to look at, there isn't anything more to do beyond creating
the
  ; columns for the fields that you want, and populating the
corresponding
  ; CDR variables in the dialplan.  For the builtin variables only, you
may
  ; create aliases for the real column name.
  ...
  [third]
  connection=sqlserver
  table=AsteriskCDR
  alias src => source
  alias channel => source_channel
  alias dst => dest
  alias dstchannel => dest_channel
  ....

So, it would appear that the methodology would go something like this:

Define your own personal CDR database, with the fields as you would wish
them.
Name the fields in such a way so as not to conflict with the existing 
CDR field names (don't use clid, src, channel, dst, etc.). I'd assume
that
if you do use the same names, it's automatically a tie from the CDR to
the DB, and you'll not be able to drop or modify those fields. So, using
different names in the DB fields would be the most general way to handle
this. You can later pretty much handle anything that might happen with
some swift, easy, config file and dialplan changes. I guess an analogy
would be patch-panels in the wiring world. Why use patch-panels? What a
waste! Think of all the money you could save if you just wired direct
end-to-end and eliminated patch-panels! Or...?

In the config file, map (alias) the usable CDR field names to the field
names you are using in your database.

In the dialplan, use the CDR(yourfieldname)=whatever to set the fields
in your database that aren't going to be set via aliasing the field
names. Do this just before you invoke something that would create a
bridge, or wherever...

You are then on the road.

my only questions are:

How do I drop builtin CDR fields? I assume all I have to do is not alias
them, or have the same name in the DB. 

===============

In the trunk version, this is pretty much how things will flow: (I
haven't finished coding this yet, but will soon).

What do I mean, I haven't finished coding this yet? What code? See
team/murf/CDRfix5.

Just before a bridge forms, the CDR structure will be updated from the
channel.
A new CDR will then be copied from this updated channel CDR. (I think
I'll remove all the cdr_reset calls everywhere else). This includes
variables. I'll call this the "bridge CDR".

Just before the bridge (conversation between two channels) forms, the
CDR start time will be set in the bridge CDR. Disposition will be set to
ANSWER, most likely. The bridge will then be formed, and the
conversation will begin. When the bridge is closed, the end time in the
bridge CDR will be set, and the bridge CDR will be posted and detached.
Your CDR backends take it from there. The start time on the channel CDRs
involved will be updated to the end time, and the channel CDR's will be
reset (answer/end times set to null).

In the case of transfers, an attempt will be made to set a new CDR field
to the uniqueID of the first channel involved in the original
conversation. This hopefully will help link the different legs involved
in the transfer. We shall see. As transfers of transfers occur, this
extra field will be propagated. But this algorithm can't cover
situations such as SIP transfers, where the phone itself mixes a 3-way--
Asterisk isn't informed of what is happening, so the linkages can't be
made. I have it operational for Zap channels.

Even in spite of some of the SIP drawbacks, at least the basics will be
recorded; for each new bridge formed, a CDR will be generated.

The NoCDR() app will be honored, but things get sticky in transfer
situations.
To entirely shut down the current CDR system, so you can use JUST your
own instead, with the existing backends, you may find yourself coding a
NoCDR() just before every dial command in your dialplan. That should
shut down transfer-generated CDRs.  At least every Dial that could be
invoked via a transfer, that is.

ForkCDR will be deprecated, and its use will generate a WARNING in the
logs. It might still work in certain situations, but I am very seriously
considering completely ripping out the "next" field in a CDR. We shall
see. I THINK we will not need it. Use CDR_CONTROL() to get the same
effect. Since CDR's generated by the CDR_CONTROL are user-generated, you
have complete control over their contents. Right now, I don't allow you
to change the start, end, or answer times-- you set those by calling
CDR_CONTROL at the appropriate moment in the dialplan, and they use the
current time. And, of course, billsec and duration time are calculated
and set when end is set, so I don't allow you to change those either,
but I don't see any compelling reasons to restrict these, and if you CDR
hackers absolutely must be able to muck with these, I will lift the
restrictions.

Under this regime, no CDR's will be generated if no conversations take
place. So, failed outgoing calls, and incoming calls that never get past
the dialplan menus will not be recorded, as well as running apps like
VoiceMail. You'll have to use CDR_CONTROL() to generate these, if you
absolutely must record such.

If and when file (Josh) completes his bridging code (see
team/file/bridging), I have been thinking about how to do CDRs in this
new universe. I think the best approach will be to store the CDR's on
the channels. Entering a bridge will update the ANSWER time on the
channel involved. Exiting a bridge will post and detach a CDR. The CDR
will bear a uniqueID we will associate with each bridge.
We may generate a special CDR for the start, answer(first channel
entering), and end times for the bridge itself. If a channel can be
attached to more than one bridge at a time, then we'll have to associate
a CDR with each channel, but have them on the bridge instead. At any
rate, it points out strategies to take with conferences in the current
regime, which I will try to attack in CDRfix5.

Ooops, pardon the brain-dump. I'm sorry. I just go and do that now and
then.

murf


-- 
Steve Murphy
Software Developer
Digium
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3227 bytes
Desc: not available
Url : http://lists.digium.com/pipermail/asterisk-dev/attachments/20070703/7f4a6ab6/attachment.bin