[Asterisk-video] mp4save(): different audio and video lengths

Tue Mar 18 06:31:35 CDT 2008

Hi Jose,

> -----Original Message-----
> From: asterisk-video-bounces at lists.digium.com 
> [mailto:asterisk-video-bounces at lists.digium.com] On Behalf Of 
> Jose M. Recio
> Sent: Tuesday, March 18, 2008 11:17 AM
> 
> Further info after more tests.
> 
> When in a H324M call and silence supression activated, 
> app_h324m() receives
> 6 AMRNB frames per second. Each frame covers 160 msec of 
> "voice". This translates into 6 RTP frames per second, each 
> one with 160 samples.
Correct.

> Each RTP frame says that it contains 160 samples, so it is 
> supposed to contain voice for 20 msec, 
Correct.

> but the receiver must 
> actually fill 160 msec with that data, as there will be no 
> more frames for 140 msec.

This I do not understand.
What is your observation.
6 AMRNB frames are announced? No, you receive 6 RTP frames?
But one RTP frame does not contain 31 Bytes, i.e. 160 samples?
Instead one RTP contains only one sample?
Something I do not understand.

If RTP PDUs are missing, then the RTP timestamp of arrived PDUs
should be properly. And then _this_ timestamp should be used to
store the frame in the mp4-file.

Then the decoder knows the gaps.

Could you shed some more light in this particular issue?

> I think the problem for mp4 will be the same, the 140 msec 
> gap must be filled.
> 
> A strategy could be replicating the frame (it contains 
> silence, with a very predictable pattern: for PCMU it is 
> "7eff7eff7e7effffffffffffffffffffffffffffffff" repeated again 
> and again) 7 times, inserting "sintetic" frames. Or make the 
> frame longer, so it lasts for 160 msec instead of 20 msec.

We should not digg into the specific payload format.
In future we could have to deal with aac, aacPlus, AMR-++ (something from
Thomson and FhG)
and if we can make it work in a general purpose way, fine.

> Sergio Garcia Murillo Enviado el: martes, 18 de marzo de 2008 2:08

> What I think it's happening is that during silence supresions 
> terminal does not send audio at all. As I calculate the 
> timestamp of the packet based on the previoud one plus the 
> number of samples, then I completelly ignore the silence 
> periods and think that both packets are continous.

I think, this is the bug. The timestamp has to be shifted to the
"calculated" value.
It is no problem to have "jumps" in the timestamps. This is the intended use
of
RTP.

> I've investigated a bit and it seems that the time difference 
> is stored in the "delivery"
> member of the ast_frame. If that's correct and also works for 
> rtp I should modify
> app_mp4 for reding the timing information from that field and 
> modify app_h324m to fill it correctly. Could anyone check if 
> this would be correct or not?
Sounds reasonable.

Greetings,
Michael

> ----- Original Message -----
> From: Jose M. Recio [mailto:josemrecio at gmail.com]
> 
> Ohhh, very interesting. I have checked and you are right ... 
> audio track length is almost zero when the handset is muted. 
> Thanks for the hint !!!
> 
> Actually I started investigating this issue trying to debug a 
> strange behaviour I had detected in RTP, and I am now sure is 
> also related to handset VAD.
> 
> When the handset is muted and VAD is activated (only 
> "silence" is sent), asterisk sends RTP packets every 160 
> msecs (as it should), but ... timestamp difference between 
> two consecutive packets is aprox. 1200-1400 msecs, not 160 
> msecs. And the marker bit is set.
> 
> When the handset is unmuted, asterisk starts sending RTP 
> packets every 20 msec, with 160 msec timestamp difference 
> between consecutive packets, and marker not set.
> 
> I can send you logs displaying this behaviour, so we can 
> further investigate.
> 
> JM
> 
> -----Mensaje original-----
> De: asterisk-video-bounces at lists.digium.com
> [mailto:asterisk-video-bounces at lists.digium.com] En nombre de Michael
> (qq12345)
> Enviado el: domingo, 16 de marzo de 2008 21:51
> Para: asterisk-video at lists.digium.com
> Asunto: Re: [Asterisk-video] mp4save(): different audio and 
> video lengths
> 
> Hi Jose,
> 
> please look in the archives. It looks like an issue of noise 
> suppression of mobile phones and therefore some mobile phones 
> do not deliver audio in that stage.
> You could help us, if you can figure out, if the uploaded 
> stream has gaps in the rtp-timestamps.
> Then we can get it back to sync.
> Another solution could be to count the missing audio frames 
> and correct the time stamp of the next available rtp audio 
> packet for storage.
> Synching of audio and video tracks is done with the 
> timestamps provided with the network PDUs.
> Perhaps the timestamps are not took into account when storing 
> the audio samples.
> 
> Your help is encouraged,
> Michael
>  
> 
> > -----Original Message-----
> > From: asterisk-video-bounces at lists.digium.com
> > [mailto:asterisk-video-bounces at lists.digium.com] On Behalf 
> Of Jose M. 
> > Recio
> > Sent: Sunday, March 16, 2008 9:35 PM
> > To: asterisk-video at lists.digium.com
> > Subject: [Asterisk-video] mp4save(): different audio and 
> video lengths
> > 
> > Hi, guys,
> > 
> > I don't know if that's a known fact, I have discovered 
> that, at least 
> > in my systems, mp4save() creates mp4files with different lengths in 
> > audio and video tracks (see examples at the end). This 
> means that the 
> > audio track in those files is almost useless, as it is 
> totally out of 
> > synch with the video ...
> > 
> > I think this has something to do with the framesPerSample 
> property in 
> > mpeg4ip package, but after messing a bit with the
> > MP4 library I have surrended without fixing it. Have 
> somebody tested 
> > or fixed this before?
> > 
> > Can you check by yourself if this behaviour also happens in your 
> > system?.
> > 
> > Thanks,
> > JM
> > 
> > 
> > ~$  mp4info /tmp/*.mp4
> > mp4info version 1.5.0.1
> > /tmp/save.clock.1.mp4:
> > Track   Type    Info
> > 1       audio   AMR, 6.780 secs, 12 kbps, 8000 Hz
> > 2       hint    Payload AMR for track 1
> > 3       video   H.263, 9.171 secs, 47 kbps, 176x144 @ 8.614110 fps
> > 4       hint    Payload H263-1998 for track 3
> > /tmp/save.clock.2.mp4:
> > Track   Type    Info
> > 1       audio   AMR, 52.460 secs, 12 kbps, 8000 Hz
> > 2       hint    Payload AMR for track 1
> > 3       video   H.263, 61.413 secs, 45 kbps, 176x144 @ 9.069741 fps
> > 4       hint    Payload H263-1998 for track 3
> > /tmp/save.clock.3.mp4:
> > Track   Type    Info
> > 1       audio   AMR, 2.900 secs, 10 kbps, 8000 Hz
> > 2       hint    Payload AMR for track 1
> > 3       video   H.263, 6.649 secs, 49 kbps, 176x144 @ 8.572718 fps
> > 4       hint    Payload H263-1998 for track 3
> > ~$
> > 
> > 
> > 
> > _______________________________________________
> > --Bandwidth and Colocation Provided by http://www.api-digital.com--
> > 
> > asterisk-video mailing list
> > To UNSUBSCRIBE or update options visit:
> >    http://lists.digium.com/mailman/listinfo/asterisk-video
> > 
> 
> 
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
> 
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-video
> 
> 
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
> 
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-video
> 
> 
> 
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
> 
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-video
> 