[asterisk-dev] Time for a bug fix phase?

[Dmitry Andrianov]

By the way, I wonder how current CDR scheme will be working. Will it be just left as is or will it be removed from the PBX core and re-implemented as a CEL event consumer?

Making it just one of CEL consumers looks a good idea to m. Doing so will also quickly identify if anything is missing in CEL design (if existing CDR cannot be written using CEL, then something is probably wrong).

Regards,
Dmitry Andrianov

-----Original Message-----
From: asterisk-dev-bounces at lists.digium.com [mailto:asterisk-dev-bounces at lists.digium.com] On Behalf Of Steve Murphy
Sent: Tuesday, June 03, 2008 8:52 PM
To: Atis Lezdins
Cc: Asterisk Developers Mailing List
Subject: Re: [asterisk-dev] Time for a bug fix phase?

On Mon, 2008-06-02 at 22:46 +0300, Atis Lezdins wrote:
> On Sat, May 31, 2008 at 10:52 PM, Steve Murphy <murf at digium.com> wrote:
> > On Sat, 2008-05-31 at 16:00 +0100, Grey Man wrote:
> >> On Sat, May 31, 2008 at 3:00 PM, Steve Murphy <murf at digium.com> wrote:
> >> >
> >> > Which version of Asterisk did you do this in? The stuff I in did
> >> > 1.4 is different than what I did in trunk.
> >>
> >> 1.4.19 but I'd be pretty confident the CDR code in res_features.c
> >> hasn't changed in later 1.4 versions.
> >>
> >> > In general, the CDR merging was necessary to combine info that
> >> > was captured in either the channel or peer CDR's.
> >>
> >> What sort of information? To me it looks like information is only
> >> going to be lost by merging. For example if the channels on either
> >> side of the bridge happen to have different accountcodes then since
> >> there is only one accountcode per CDR a combined CDR has to choose
> >> one of the accountcodes. Which happens to be the crux of the
> >> problem when an attended transfer takes place.
> >>
> >
> > Well, it was pretty random as to what information exactly.
> > The main fields that really concerned me were the start, answer, and
> > end times. Call disposition.
> >
> > As to disambiguation, you are right, and it *is* a problem, when
> > both CDR's have a chunk of info in their respective structs, that
> > contradict one another. In which case, we
> > *try* to pick the one that makes the most sense, somehow.
> > Usually, the channel (as opposed to the peer) would have been the
> > one to go out; for times, the earlier start time is probably the
> > best. The later end time. And so on.
> >
> > Of course, in the 1.2 world, the data in the peer CDR never even got
> > set-- because the peer's CDR wouldn't be allocated. So, you just had
> > to settle for what you got.
> >
> >
> >> > And also, in general, I find that the CDR system is pretty much
> >> > smeared across the entire code base, and innocuous changes in one
> >> > part can solve one problem, but create a cascade of problems in other areas.
> >>
> >> I don't doubt you there. That's one reason why I think the problem
> >> needs some thought put into the design instead of just jumping in.
> >> The starting point for a CDR design that works correctly should be
> >> to create an option that allows the whole bridged/merged CDR
> >> approach to be turned off and instead get a CDR per channel.
> >>
> >
> > I agree that we do need to think about how Asterisk should evolve
> > its CDR approach.
> >
> > And you are, in a kind of way, pointing out a good approach, in that
> > changes should be optional to the community.
> >
> > But it does get a bit ugly in regards to pleasing the community wrt
> > CDRs... And here's what worries me: I have no clear perception that
> > the community would be happy with one single CDR approach.
> > What I'm perceiving is quite the opposite: different 'users'
> > have quite different concepts of what they want the CDR system to
> > report.
> >
> > For instance, Brian was quite unhappy with the direction I took in
> > 1.4, with (attempting to, at least) produce one CDR per bridge.
> > He's more interested in 'logical' conversations, and the amount of
> > time they take. I just have trouble coming to grips with what that
> > 'logical' conversation consists of...
> >
> > So, CEL seemed the perfect answer to me. You basically drop the CDR
> > concept entirely, and go down a level to the individual events from
> > which CDR's are usually compiled. You get a timely notification of
> > every channel creation (start), answer, hangup (end), every transfer
> > request, park, unpark, conference join, and so on, and from those,
> > YOU decide how and what to do with each event.
> >
> > At first, I thought just databasing these and running a
> > post-processor on the data would give most people what they need.
> > But Brian pointed out that such a database would not be that useful.
> > Queries for common and useful chunks of info would be exceedingly
> > complex if possible at all. So Brian proposed that a CEL -> CDR
> > converter would be in order, which I thought was a BRILLIANT idea.
> > His needs later pointed him in other directions... but the idea remains.
> >
> > Why do I think it's so brilliant? Well, right now CDR code is spread
> > out all over Asterisk. With a CEL-> CDR converter, all the code to
> > collect data and emit CDR's would be in a single source file (or a
> > set of them, depending on the code size and organization level of
> > the author). True, the CEL calls would be spread out over all
> > creation, but that's OK.
> >
> > CEL calls look a bit like Manager event calls in the source, and in
> > a lot of cases, are called from the same places. I thought about
> > just using manager calls, but they have a sufficiently different set
> > of requirements, to justify a separate interface (at least, for the
> > time being). The Manager interface is a socket with text messaging,
> > the manager events cover a much larger scope of events, and so on.
> > Perhaps in the future, we can unify the undercarriages of Manager
> > and CEL.
> >
> > So, with CEL, there's no need to get together and decide on policies.
> > The only problem that the different CDR camps could have with CEL is
> > that it may not provide some camps with enough events. That should
> > be no problem-- those who don't need to know about some events can
> > simply ignore them.
> >
> > You see, the CDR system was meant to be a source of TRUTH, from
> > which all could put in a spoon and scoop out the facts that they
> > need for billing. But as it turns out, it isn't such a great oracle:
> > it's not only stating some facts (but not all), and it's also giving
> > us interpretation of the facts. Sometimes not useful interpretations.
> >
> > Everyone thinks that CDRs alone can give us a perfect CALEA memory
> > of the Truth of what happened. But it can't. And everyone expects
> > that they can with a little (hopefully no) post-processing, they can
> > generate billing statement directly from the CDR's: every CDR is a
> > billable statement of some kind. But that's not true, either.
> >
> > Forming a CDR is a result of individual needs and choices.
> > If all that everybody did was to pick up a phone, dial a number,
> > talk, and hang up, well, that's a no-brainer, CDR's cover this
> > situation 100% and that's that. But if we rang a phone and no-one
> > answered, do want a CDR for that? If we rang a dozen phones, do we
> > want to know about only the phone that got answered first?
> > If we park someone, do we care how long they were parked? Do we care
> > how long people were kept on hold?  If we forward a call and
> > participate in 3-way conferences, does anyone care about that?
> > If we forward to an extension in the pbx, do we care? if we forward
> > to a long-distance number across the world, who gets billed for that?
> > Or do we care? What events are we legally required to track? If we
> > use a local channel to play a recording, do we charge for that?
> >
> > My dream is produce a general purpose CEL->CDR converter, from which
> > perhaps you could use a config file to describe which kind of events
> > you want CDRs to describe, and how... I'm sure it could have more
> > options than the Dial app.
>
> Hello,
>
> All this CEL stuff sounds great, it seems to overcome something I have
> written using CDR manipulation in dialplan now. However, I currently
> don't have big picture in my mind regarding how it would actually
> work.
>

That reminds me-- I promised to provide an app that will allow you to insert an event into the CEL event flow...

> However, i wonder won't this config approach be somehow limited? Often
> there are data you want to log to sepcific record directly from
> dialplan. So, this is idea that has been long in my mind (since I
> started messing around with CDRs), but now I feel that I can express
> that, because I have learned a little about channels, bridges and how
> do they work.
>
> So, what I've been missing since beginning is the way to control
> second channel in a call. For example when I do Dial() another channel
> is created, and if called peer answers they are bridged. So, the big
> idea is that every such channel could be executing some part of
> dialplan, so user can decide there if CDR for this channel should be
> written, from which point, and what else to add to that CDR.

As to executing dialplan code on the 'peer' channel involved in a Dial(), if you search the Dial() options, you will see that you can execute a macro/gosub on the second channel at certain times. Refer to the Dial documentation.

CEL has nothing directly to do with CDR's. But, one possible approach would be to shut off the normal CDR system in Asterisk, and use your own CEL backend to generate CDRs the way you need them to be. Exactly the way the you need them to be. So, if certain properties of a channel are right, or certain events occur in the life of a channel, you can skip generating CDRs, or generate extra CDR's, to your heart's content.

I'm out of new-feature-development time for a while, but when I've regained my new-feature-development karma, I might write an example CEL backend to do that.
In the meantime, if anyone is piqued enough to want to do this, I humbly bow to their wishes, and I might be good for a bit of advise here or there.

murf

--
Steve Murphy
Software Developer
Digium