[asterisk-users] Simple CDRs

Thu Jan 8 12:22:16 CST 2009

On Thu, 2009-01-08 at 16:20 +0000, Grey Man wrote:
> On Wed, Jan 7, 2009 at 6:01 PM, Steve Murphy <murf at digium.com> wrote:
> > On Wed, 2009-01-07 at 02:56 +0000, Grey Man wrote:
> >> On Tue, Jan 6, 2009 at 3:53 PM, Steve Murphy <murf at digium.com> wrote:
> >>
> >> That sounds a bit dangerous to me. If you go down the path of setting
> >> the answer time based on dial plan applications or events you'll need
> >> to understand and modify every dial plan application that can answer a
> >> call. To me it would seem a lot simpler to do the
> >> override/modification in each channel or even better even lower in
> >> ast_channel. A channel has to have a very clearly defined definition
> >> of answer and hangup whereas dial plan applications don't.
> >>
> >
> > Hadn't thought along that line before: classifying the *kinds* of
> > Answer.
> > But we may not have to bother.... In this mode, a single CDR could
> > cover several dial attempts and their corresponding "answers"....
> > incoming caller A could, during his lifetime in Asterisk, dial
> > several users, say via assisted xfers, and have the targets hang up
> > first, leaving his channel intact. It doesn't make sense to store an
> > outgoing dial result in the originating channel CDR....
> 
> A single CDR should not be able to cover several Dial attempts or even
> multiple destinations within the same Dial attempts. If you think of
> Dial as just another application and of CDRs being designed to reflect
> the lifetime of a channel then things are simplified. If a Dial or any
> other dial plan application causes three channels to be created then
> that's 3 CDRs that will be generated. If the first Dial call fails and
> another one is made with another 3 destinations than that's another 3
> CDRs that will be generated.
> 

OK.

> > What if, instead, we record the disposition of the dial in the created
> > channel CDR? A channel comes into existence only once. For external
> > incoming
> > calls into the PBX, the disposition is pretty uninteresting, because
> > we'd
> > not create the channel if the disposition wasn't ANSWERED... (well, at
> > least
> > in the DAHDI world, maybe not so much in the voip world)... but,
> > once asterisk dials B in A's behalf, we could record all the dialing
> > action
> > in the CDR for B; It's src/dest related fields could record the dial
> > from
> > A to B; the disposition would be FAIL/BUSY/NOANSWER/etc...
> 
> That's not quite true. An incoming call to Asterisk could FAIL due to
> some dialplan condition, it could go UNANSWERED if the dialplan
> forwards the call to a destination that does not pick up, it could get
> CANCELLED by the external party before Asterisk finishes processing
> etc.
> 
Aha! Hadn't thought on these lines... but that's true. Dahdi is fairly
binary, but SIP has more possibilities.

> >> > Oh, and BUSY/NO ANSWER/FAIL for a non-s exten, would also override
> >> > an ANSWER on exten s, BTW...
> >> >
> >> > And, would it be proper to include all dial attempts? My guess is
> >> > that you would *NOT* want to see any dial attempts in this mode. Well,
> >> > at least, in this particular case, if A *tries* to dial B, but B
> >> > doesn't answer, then since A is a live channel, we would record
> >> > it's life in the system. When A hangs up, we would see the NO ANSWER
> >> > disposition, and the destination of B, right? If A tried to dial a
> >> > group, and nobody answered, the destination would be a random member
> >> > of that group, the args to the Dial command would record the other
> >> > members, usually.
> 
> If A tried to dial a group that consisted of 10 members then that's 10
> CDRs that should be generated. For those people that don't want
> unanswered CDRs they can be easily turned off with the configuration
> option. For those of us that want a CDR for every call which includes
> successful and unsuccessful call attempts we ge them.
> 

OK. It's looking like we are the same wavelength.

> > Let's say you do your examp, but we add exten dev SIP/w to the
> > list for variety;
> > suppose:
> >
> > W just plain doesn't ANSWER
> > X is offline, so it's FAIL;
> > Y gets answered;
> > Z is busy
> >
> > If you have unanswered = yes, then you'd expect to see:
> >
> > 1. A to Asterisk which is answered (by the dialplan)
> > 2. Asterisk to X which is FAIL
> > 3. Asterisk to Y which is answered,
> > 4. Asterisk to Z which is BUSY
> > 5. Asterisk to W which is cancelled
> >
> > But if you have unanswered = no, then I guess you'd just see:
> > 1. A to Asterisk which is answered (by the dialplan),
> > 2. Asterisk to X which is FAIL
> > 3. Asterisk to Y which is answered,
> > 4. Asterisk to Z which is BUSY
> >
> > or would unanswered = no also suppress #2 and #4?
> 
> I think with unanswered=no the desire would be to exclude any call
> with billsec=0. The disposition field is a descriptive field and not
> something that I think should be relied upon too heavily. Different
> channels may set the disposition field differently for eaxmple the SIP
> channel could get clever and set it to  "FAILED - 403 Forbidden" to
> reflect the SIP response code that caused the failure. That
> description would obviously not make sense for DAHDI.
> 
> I don't think it should be the case but if it looks like disposition
> is increasing the complexity of the design or implementation it could
> probably be dropped as it's not a critical CDR field and CEL will
> provide more information in that area anyway.
> 
At the moment, disposition is an integer with values defined in
cdr.h; The values are carefully sorted to give a numerical ranking
that ends applying to a group of targets, for a single result that
would normally make sense. Ie, if 3 targets are specified for a call,
and one is FAIL (unplugged, offline, whatever), another is BUSY, and
the third ends up NOANSWER, then NOANSWER wins for the group as a
response... 

The upshot is that up to now, disposition wasn't stored as string, so
no neat new stuff could be stored there. Indeed, any vals will have to
be carefully weighed and engineered into the current design, to give
the same behavior as we see now... the new cel2cdr implementation is
wide open, tho, and we can spec it the way we *want* it to be.

billsec==0 is a pretty easy heuristic for unanswered=no to filter on...

> >> > OK, gotcha. Now, let's talk about the fields in current/future CDRs,
> >> > and see which you consider relevant?
> >>
> >> The core fields I would put into the Asterisk CDRs are:
> >>
> >> - uniqueid: A GUID/UUID that cannot be changed and is critical for billing,
> >
> > In my previous message, I proposed that we could add some config options
> > that would define uniqueid/linkedid behavior, let's get explicit:
> >
> > unique_append_ident=<string>
> >
> > The <string> would be appended to the unique id/linked id when it
> > goes "out the door"; This would allow you to tack on a system name
> > or ip or whatever to every uniqueid/linkedid as it goes into the db/log
> > file/whatever; if you had several systems logging to the same db,
> > you could guarantee that the id's were unique across systems that way.
> >
> > unique_append_uuid = yes
> >  would append the uuid (all 36 bytes of it)
> >  to the uniqueID/linkedid. This uuid is calculated once per Asterisk
> >  invocation;
> >
> > unique_append_uuid = per_channel
> >  would append the uuid (all 36 bytes of it)
> >  to the uniqueID/linkedid. This uuid is calculated once per
> >  channel created.
> >
> > Will this suffice?
> 
> I guess so but that seems to be a bit of overkill. The probability of
> a UUID clash are some astronomical figure so why not just generate the
> UUID when the CDR is created. I don't know what exactly what
> conditions you'd need to get such a conflict but at a minimum I
> suspect it's two servers with identical MAC addresses that generate
> the UUID at exactly the same tick on their real-time clocks. You'd
> have to work pretty hard to get those conditions and in my opinion
> appending a UUID to a UUID is not needed, a single UUID per CDR would
> be sufficient.

Well, as I explained earlier, yes, we could do it that way, but the
current linkedid is based on uniqueID, and the magic of it is that
can thus determine "older" from "younger" because of the timestamp
base. So, the linkedID is locked to that sort of framework until 
someone can figure out a better method. Tacking on something else
is purely to disambiguate between different systems.

I really have no idea how uuids were engineered. If I were doing it,
I'd use some sort of random number generator... but I didn't do it.
I'd hope it's entropy based somehow and the likelihood of two
uuid requests occurring exactly at the same time, generating the same
id would be astronomically small, as small as ANY two uuids colliding.

> 
> >> - calldirection: 0 for a call Asterisk receives and 1 for a call
> >> Asterisk initiates,
> >
> > Interesting. How exactly do we calculate this? How do we tell
> > an incoming FXO call from an incoming FXS call? The FXS lines
> > will get special attention from asterisk; dialtone is generated,
> > we are waiting for an extension in the dialplan. FXO calls
> > start at the 's' extension... the channel driver, I guess, knows
> > the diff... but higher up?  And I've been reminded that the
> > FXO isn't automatically mean that the call is incoming... and
> > vice-versa,
> > so... this is logic external to CDRs, methinks? As to
> > SIP, I guess anything that's not a SIP peer (via sip.conf) is an
> > incoming call? All the time? Guaranteed?
> 
> The key there is exactly as you mention: the channel driver knows.
> Keeping the CDRs away from the dialplan and the s extension etc. are
> what's needed here. However like disposition this is not a critical
> field and if it gets too complicated could be left out.

In general, stuff like FXO's activating, would signal an incoming call,
but.... gee, I could wire up some odd circuit to make it a local
extension...
Some external rule to the pbx determines which devices and peers you'll
want to pay (billable) attention to. Don't you think?

> 
> >> - accountcode (user modifiable)
> >> - clid: The channel identifier of the call originator equivalent to
> >> the A number on a traditional telco CDR,
> >
> > Actually, the underlying CEL interface will report ANI, DNIS, etc.
> > so all these fields are available. I think it would be wise to
> > report (or at least allow) all these values.
> >
> >> - dst: The destination of the call equivalent to the B number on a
> >> traditional telco CDR,
> >
> > I think this is the destination exten.
> >
> > You might want to add the channel, dstchannel, which record the
> > originator/target device info.
> 
> Yep it it's easy extra info like that can only be useful.
> 
> >> - starttime: The time Asterisk first receives or initiates a call,
> >
> >
> >> - progresstime: The time Asterisk first receives or generates a
> >> progress indication,
> >
> > This is useful? I'd have to add this every driver; some may not
> > provide it (maybe)... this is pretty deep-down stuff. How is it
> > useful?
> 
> Again not an essential field but it's potentially very useful.
> Consider a provider sending calls via a wholesale PSTN gateway it it's
> taking 10s to get back the first info response (100 Trying, 180
> Ringing. 183 Progress) then they are going to be giving their
> customers a bad experience. Getting easy visibility into that time
> would be very useful to me.
> 
> >> - answertime: The time a call is answered,
> >> - endtime: The time a call is hungup or cancelled,
> >> - duration: endtime - starttime,
> >> - billsec: endtime - answertime,
> >> - disposition: ANSWERED, TIMEOUT, CANCELLED, HUNGUP and maybe others,
> >
> > ANSWERED, NO_ANSWER, FAILED, BUSY, CANCELLED I understand,
> > but you'll have to explain HUNGUP...
> 
> Your'e right HUNGUP doesn't make sense it would mean the same as ANSWERED.
> 
> > While we are at it, how about hangupcause? Hangup originator?
> 
> Yes they would be useful but in my SIP experience those fields are not
> provided by that many user agents so would probably be emtpy 90% of
> the time. Personally I wouldn't mind having them included.
> 
> >> - userfield (user modifiable): General purpose field for any custom
> >> CDR info needed by Asterisk users.
> >>
> >> Some extra fields that I think would also be very (if not very very)
> >> useful to people:
> >>
> >> - remoteip: The remote IP address of the call where relevant. For an
> >> incoming call the originator's IP address, for an outgoing call the
> >> desitanation IP address.Apart from being very useful information it
> >> would also be one piece of information useulf in identifying brute
> >> force or DoS attacks,
> >>
> >
> > This would be good info from the voip related protocols... it seems
> > reasonable to add this, but I kinda wonder how hard this will be to
> > coax from the channel drivers... I have to look and see if this stored
> > on the channel... never played with it before.
> >
> >> - linkedid: Where there is a logical link between different calls such
> >> as an incoming call generating an outgoing call this field could be a
> >> common UUID/GUID.This field is not as important as the core ones so if
> >> there are situations where it becomes ambiguous or doesn't make sense
> >> it could be ignored,
> >>
> >> - channel: The data that was passed to the channel driver for a call.
> >
> > you probably mean the lastapp/lastdata fields here. I think we may
> > need to look at these fields a little closer. There are several places
> > in the code where a dial-type-operation is performed, but the Dial() app
> > is not called to do it... followme, queues, call files, ami originate,
> > places like that. It might be good to have the method of making the
> > call recorded in the CDR in the lastapp field; like queue#Dial
> > park#Dial,
> > CallFile#Dial, etc.
> >
> > Also, you left out the "party" field... If A calls in, and dials B,
> > (well, asterisk dials B in A's behalf), and B xfers A to C, then,
> > wouldn't you be at all interested in the fact, that even tho B called
> > C, it was A that spent all the time on the line? Eh, I guess it wouldn't
> > really matter, would it? If C was in Namibia, would B always be billed
> > for the call, or would anyone ever be tempted to bill A's dept for that
> > call? (Assuming, I guess, that A & B are both internal)...
> 
> I was thinking more of the piece of data passed to the channel driver
> and not the data provided to any particular dialplan application. It
> gets back again to CDRs reflecting channel activity and not dialplan
> application activity. The current Asterisk CDR situation illustrates
> the problem of moving the CDRs up to the dialplan application layer,
> the solution is to sink them back down to the channel layer.
> 
> At the end of the day a Call Detail Record by definition should be
> recording call activity. In Asterisk it's the channels that end up
> making and ending the calls and not the dial plan or it's applications
> hence it's the channels that are the best place to generate the CDRs.

I take that as a "heck no, I'm not interested in the slightest
about that!" ;)

As to the current CDR situation, it is where it is, because
of several factors; like a large number of users with vastly different
desires and demands; The cdr is stored on channels; several of the
CDR fields are copies of channel fields, and no spec to define
where/when the proper updates are supposed to be done; CDR fields
are updated and set at various places and times with no development
documentation as to why; CDRs depend on "channel/peer" roles that
channels take on, but that some operations swap around; 
and on and on... I have never seen a mechanism so intertangled 
and interdependent... It's screaming "rewrite me!".

I think I have enough to spec this mode now.

OK, I'll take a few days and add this to the RFCs/CDRfix2.txt document,
and maybe folks could review it and find the holes/flaws. I'll try and
throw in some examples, etc.

The sooner I have a working implementation, the sooner I can 
start deprecating the current stuff.

murf

> 
> Regards,
> 
> Greyman.

-- 
Steve Murphy <murf at digium.com>
Digium
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3227 bytes
Desc: not available
Url : http://lists.digium.com/pipermail/asterisk-users/attachments/20090108/c950d320/attachment.bin 