[Asterisk-Dev] IAX2 Receiver Reports

[Steve Kann]

On Dec 29, 2004, at 8:52 PM, John Todd wrote:

> At 12:30 PM -0500 on 12/28/04, Steve Kann wrote:
>> A proposal for IAX2 receiver reports:
>>
>> The idea of receiver reports is to allow call quality monitoring for 
>> IAX2 calls.  By having receivers monitor quality (presumably through 
>> a jitterbuffer), and report network conditions back to senders, each 
>> side of the call should have a relatively complete picture of what 
>> the entire call looks like.
>>
>> Presently, IAX2 gets a good idea of network Round-Trip time by 
>> sending PING frames to the remote side, and getting PONG frames back.
>>
>> Also, LAGRQ frames are presently sent to the remote side, in order to 
>> get an idea of the Round-Trip + playout(jitterbuffer) delay.  The 
>> implementation of LAGRQ is presently a bit broken, because they 
>> currently go through the jitterbuffer on _both_ the sender and 
>> receiver side, and the timestamps used to do this are wrong. 
>> (actually, I haven't checked this recently, but it was the case).
>>
>> I propose:
>>
>> 1) Eliminating LAGRQ frames altogether.  I think this system will be 
>> more accurate anyway.
>>
>> 2) Adding the following IEs to PONG frames returned in response to 
>> PING frames (or, we could add this information to the PING frames; it 
>> doesn't matter, really:
>>
>> #define IAX_IE_RR_JITTER                        42              /* 
>> Received jitter (as in RFC1889) u32 */
>> #define IAX_IE_RR_LOSS                          43              /* 
>> Received loss (high byte loss pct, low 24 bits loss count, as in 
>> rfc1889 */
>> #define IAX_IE_RR_PKTS                          44              /* 
>> Received frames (total frames received) */
>> #define IAX_IE_RR_DELAY                         45              /* 
>> Max playout/jb delay for received frames (in ms) u16 */
>>
>>
>> The Jitter IE would be the presently measured jitter.  I've defined 
>> it here to be compatible with the RFC1889 (RTCP) measurement, but the 
>> mechanics of measuring it aren't that important, as long as it is 
>> measured in milliseconds.
>>
>> The Loss IE contains two numbers packed in a 32 bit field, as in 
>> RFC1889.  The high 8 bits represent the percentage loss since the 
>> last RR was sent (i.e. the last PING or PONG).   The low 24 bits 
>> represent the total number of lost frames since the beginning of the 
>> call.  (this number may go down, of course, if packets presumed lost 
>> are received after the RR is sent).
>>
>> i.e.:  IAX_IE_RR_LOSS = ((loss_pct & 0xff) << 24) & (loss_cnt & 
>> 0x00ffffff);
>>
>> IAX_IE_RR_PKTS is the total number of IAX2 packets received during 
>> the present IAX2 call.  (therefore, loss_cnt / IAX_IE_RR_PKTS is the 
>> fraction of lost packets for the whole call).
>>
>> IAX_IE_RR_DELAY is the maximum playout(jitterbuffer) delay that a 
>> received frame is expected to face, given that the received frame 
>> falls into the present jitter window.   A sender should be able to 
>> report that the total one-way "lag" between when it injects an audio 
>> frame into the network, and when it is rendered at the receiver's 
>> side, is less than or equal to the calculated round-trip time, plus 
>> this delay.  (Generally, it would be less than or equal to 1/2 RTT 
>> plus this delay, but it's difficult to determine one-way transit 
>> time).
>>
>> ================
>>
>> How this information may be used:
>>
>> An endpoint would then have the following useful information;
>>
>> Jitter[In/Out], Loss[In/Out], Lag[In/Out], RTT
>>
>>
>> You could store this information, and monitor it, to determine if you 
>> have a network problem.  If you had many calls running 
>> simultaneously, you could tell if the problem was yours (seems to 
>> affect all calls), or perhaps not (only affects a few calls).  You 
>> could tell if the problem was increasing jitter, increasing RTT, or 
>> packet loss.
>>
>> I plan on adding this to libiax2 soon as a test, but to be useful, it 
>> would need to be sent and interpreted on the chan_iax2 side as well.  
>> Also, it would be nice to send and interpret the RTCP-compatible RR 
>> information in RTCP from rtp.c as well.
>>
>> Comments are welcome.
>>
>> -SteveK
>
> Steve -
>   This all looks very good.  The "service provider" portion of the 
> community should be falling over themselves to get this kind of stuff 
> implemented, but there are apparently two camps: the camp that is too 
> resource/cash/staff poor to implement the measurement and monitoring 
> tools for this (thus rendering it low on the priority list) and the 
> camp that has more money than they know what to do with and they go 
> right for the very expensive tools which use RTCP.  Of course, I'm 
> speaking of SIP tools right now; it would be nice to also include some 
> RTCP-like tools inside of IAX2 that can be measured easily and built 
> right into Asterisk as some variant of a CQDR log.  (Call Quality 
> Detail Record.)
>
>   Cisco has implemented in their 79xx SIP images a fairly interesting 
> feature which transmits some RTP statistics in the BYE message. Then, 
> each "hop" in the SIP tree can collect and record stats about the 
> end-to-end transaction.  I don't know how one would go about 
> correlating media streams where multiple media endpoints are concerned 
> (think "IAX2 border session controller") but that seems to become more 
> of a database relationship question than one of protocol messaging, 
> though one could possibly force the issue into the protocol (gah.)
>
>   I would humbly suggest the following additions or expansions of the 
> data collected:
>     - Lost packets

Lost packets are already identified in the above.

>     - Out-of-order packets

Based on what you're writing below about this, I think what you want is 
not "out of order" packets, but "dropped" packets;   Out-of-order 
packets aren't a problem when you have a jitterbuffer; it puts them 
back in order without a problem.

Dropped packets, however, occur (in a simplified explanation) when a 
packet arrives too late to play.  This occurs whenever the jitterbuffer 
cannot anticipate, based on it's historical view of arrival times, that 
a packet may arrive at such a time.    Because it is so related to the 
algorithms (and settings) of the jitterbuffer itself, it is not 
necessarily an indicator of the network condition as it is the 
jitterbuffer implementation.

Given those explanations, "jitter" is a number which describes the 
network condition best, but we could also actually count out-of-order 
packets, and I already do internally keep track of dropped packets in 
the jitterbuffer.

>     - Octets

We could also keep track of this, but the question would then be what 
do you want to measure?  There's a bunch of decisions:

1) All frames, or just media (audio+video) frames.  (to be comparable 
to RTP, you'd just want media, but to look at the whole stream, you'd 
want everything, including control frames that do and do not pass 
through the jitterbuffer.

2) Which parts of the packets should we count?  The payload (media 
data) [obviously], IAX2 overhead [small].  Those can be measured 
easily.  IP [large] and UDP [not quite so large] overhead can be 
guessed, but it's not easy to get that from the stack in a 
platform-independent and non-root-needing way.  Ethernet overhead?  [14 
bytes/packet]  Ethernet headers make up more than 5.6kbps of traffic 
for a 20ms/frame audio stream, but this framing is reduced when being 
sent over a WAN link with smaller framing (i.e. PPP).


>     - Codec

This can, of course change during the call, but IAX2 already knows this.

>     - Number of seconds in media stream (as of that report interval)

We already know this as well..


>
>   Having both the lost packets and out-of-order packets can possibly 
> be recorded as independent values with only a little effort, and 
> having some information about each would greatly enhance debugging. 
> Adding lost packets to out-of-order packets would result in the total 
> "unplayed packets".  I know that out-of-order frames are a big problem 
> in certain exotic environments (poorly balanced but equally weighted 
> connections upstream or downstream) and having some additional 
> datapoints to work with would probably be useful to the VOIP 
> administrator when tracking down problems with poor-quality 
> connections, especially in international situations where often there 
> are multiple-path connections that are poorly configured.

Sure, but then you'd see high jitter, and the jitterbuffer would adjust 
to compensate, and the result would just be additional latency;

>
>   Additionally, you may want to make the codec available and the 
> duration of the RTP stream up until that point, as well.  While this 
> can be determined perhaps from other logs, it might be handy to have 
> it presented in the QoS log as well instead of trying to scrape it 
> from somewhere else.  At the worst, it can just be discarded.
>
>   I suspect that most admins would want to store (typically) the last 
> value set recieved and generated, but a "live" monitor would be really 
> great to build into softphones so that a little control panel could be 
> popped open to watch real-time network conditions as a tech works with 
> the audio path on the network side...
>

Right, this is what iaxclient will enable authors to do;  I presently 
have a test client which logs the information proposed above to stderr. 
  (Presently, the RR stuff I put into chan_iax2 only gives you jitter, 
because the other information isn't known (loss), or isn't easily 
accessible (delay).

I could add "dropped" easily, and out-of-order would probably just be a 
few lines as well..  It isn't a problem to do, but all these numbers 
could just end up being confusing for people.

(actually, I think, as long as the path latencies are close to equal 
(i.e. plus or minus 20-30ms of each other), using multipath links is 
can be helpful if there are lossy conditions, because it will make the 
resultant stream interleaved, such that it is less likely to result in 
consecutive lost frames, which makes interpolation (PLC) more 
effective.

-SteveK