[Asterisk-video] mp4save(): different audio and video lengths
Michael (qq12345)
qq12345 at web.de
Tue Mar 18 08:10:42 CDT 2008
Hi,
ok. Thanks for your explanation.
So, we can do the following trick:
reuse the last frame (assuming it is silence or take your generic silence
frame)
and add it multiple times to the track.
The result should then be, that there are continous PDUs in the
audio track without any gap.
We could change the rtp-track timestamps, too. My first intention. But my
colleague told me, that currently all video/audio decoders/players
assume a continous flow of audio frames. So the practice wins....
Can you help in modifying the code?
Greetings,
Michael
> -----Original Message-----
> From: asterisk-video-bounces at lists.digium.com
> [mailto:asterisk-video-bounces at lists.digium.com] On Behalf Of
> Jose M. Recio
> Sent: Tuesday, March 18, 2008 1:27 PM
> To: 'Development discussion of video media support in Asterisk'
> Subject: Re: [Asterisk-video] mp4save(): different audio and
> video lengths
>
> Let me explain in another way:
>
> - When silence detection is not activated.
> Frames arrive every 20 msecs, containing 160 samples, so
> containing voice for 20 msecs.
> Frame sequence numbers are incremented by 1.
> RTP timestamps are incremented by 160 (number of samples in
> each frame).
>
> - When silence detection is activated.
> Frames arrive every 160 msecs, containing 160 samples, so
> containing voice for 20 msecs.
> Frame sequence numbers are incremented by 1.
> RTP timestamps are incremented by 1280 ( = 8*160).
>
> So, with silence activated, the system receives 20 msec of
> voice every 160 secs.
> There is a 140 msecs gap that must be filled or signalled
> some way, but currently is not.
>
> Hope it is more clear now.
>
>
> -----Mensaje original-----
> De: asterisk-video-bounces at lists.digium.com
> [mailto:asterisk-video-bounces at lists.digium.com] En nombre de Michael
> (qq12345)
> Enviado el: martes, 18 de marzo de 2008 12:32
> Para: asterisk-video at lists.digium.com
> Asunto: Re: [Asterisk-video] mp4save(): different audio and
> video lengths
>
> Hi Jose,
>
> > -----Original Message-----
> > From: asterisk-video-bounces at lists.digium.com
> > [mailto:asterisk-video-bounces at lists.digium.com] On Behalf
> Of Jose M.
> > Recio
> > Sent: Tuesday, March 18, 2008 11:17 AM
> >
> > Further info after more tests.
> >
> > When in a H324M call and silence supression activated,
> > app_h324m() receives
> > 6 AMRNB frames per second. Each frame covers 160 msec of
> "voice". This
> > translates into 6 RTP frames per second, each one with 160 samples.
> Correct.
>
> > Each RTP frame says that it contains 160 samples, so it is
> supposed to
> > contain voice for 20 msec,
> Correct.
>
> > but the receiver must
> > actually fill 160 msec with that data, as there will be no
> more frames
> > for 140 msec.
>
> This I do not understand.
> What is your observation.
> 6 AMRNB frames are announced? No, you receive 6 RTP frames?
> But one RTP frame does not contain 31 Bytes, i.e. 160 samples?
> Instead one RTP contains only one sample?
> Something I do not understand.
>
> If RTP PDUs are missing, then the RTP timestamp of arrived
> PDUs should be properly. And then _this_ timestamp should be
> used to store the frame in the mp4-file.
>
> Then the decoder knows the gaps.
>
>
> Could you shed some more light in this particular issue?
>
> > I think the problem for mp4 will be the same, the 140 msec
> gap must be
> > filled.
> >
> > A strategy could be replicating the frame (it contains
> silence, with a
> > very predictable pattern: for PCMU it is
> > "7eff7eff7e7effffffffffffffffffffffffffffffff" repeated again and
> > again) 7 times, inserting "sintetic" frames. Or make the
> frame longer,
> > so it lasts for 160 msec instead of 20 msec.
>
> We should not digg into the specific payload format.
> In future we could have to deal with aac, aacPlus, AMR-++
> (something from Thomson and FhG) and if we can make it work
> in a general purpose way, fine.
>
>
> > Sergio Garcia Murillo Enviado el: martes, 18 de marzo de 2008 2:08
>
> > What I think it's happening is that during silence
> supresions terminal
> > does not send audio at all. As I calculate the timestamp of
> the packet
> > based on the previoud one plus the number of samples, then I
> > completelly ignore the silence periods and think that both
> packets are
> > continous.
>
> I think, this is the bug. The timestamp has to be shifted to
> the "calculated" value.
> It is no problem to have "jumps" in the timestamps. This is
> the intended use of RTP.
>
> > I've investigated a bit and it seems that the time difference is
> > stored in the "delivery"
> > member of the ast_frame. If that's correct and also works for rtp I
> > should modify
> > app_mp4 for reding the timing information from that field
> and modify
> > app_h324m to fill it correctly. Could anyone check if this would be
> > correct or not?
> Sounds reasonable.
>
> Greetings,
> Michael
>
> > ----- Original Message -----
> > From: Jose M. Recio [mailto:josemrecio at gmail.com]
> >
> > Ohhh, very interesting. I have checked and you are right ...
> > audio track length is almost zero when the handset is muted.
> > Thanks for the hint !!!
> >
> > Actually I started investigating this issue trying to debug
> a strange
> > behaviour I had detected in RTP, and I am now sure is also
> related to
> > handset VAD.
> >
> > When the handset is muted and VAD is activated (only "silence" is
> > sent), asterisk sends RTP packets every 160 msecs (as it
> should), but
> > ... timestamp difference between two consecutive packets is aprox.
> > 1200-1400 msecs, not 160 msecs. And the marker bit is set.
> >
> > When the handset is unmuted, asterisk starts sending RTP
> packets every
> > 20 msec, with 160 msec timestamp difference between consecutive
> > packets, and marker not set.
> >
> > I can send you logs displaying this behaviour, so we can further
> > investigate.
> >
> > JM
> >
> > -----Mensaje original-----
> > De: asterisk-video-bounces at lists.digium.com
> > [mailto:asterisk-video-bounces at lists.digium.com] En nombre
> de Michael
> > (qq12345)
> > Enviado el: domingo, 16 de marzo de 2008 21:51
> > Para: asterisk-video at lists.digium.com
> > Asunto: Re: [Asterisk-video] mp4save(): different audio and video
> > lengths
> >
> > Hi Jose,
> >
> > please look in the archives. It looks like an issue of noise
> > suppression of mobile phones and therefore some mobile
> phones do not
> > deliver audio in that stage.
> > You could help us, if you can figure out, if the uploaded
> stream has
> > gaps in the rtp-timestamps.
> > Then we can get it back to sync.
> > Another solution could be to count the missing audio frames and
> > correct the time stamp of the next available rtp audio packet for
> > storage.
> > Synching of audio and video tracks is done with the timestamps
> > provided with the network PDUs.
> > Perhaps the timestamps are not took into account when storing the
> > audio samples.
> >
> > Your help is encouraged,
> > Michael
> >
> >
> > > -----Original Message-----
> > > From: asterisk-video-bounces at lists.digium.com
> > > [mailto:asterisk-video-bounces at lists.digium.com] On Behalf
> > Of Jose M.
> > > Recio
> > > Sent: Sunday, March 16, 2008 9:35 PM
> > > To: asterisk-video at lists.digium.com
> > > Subject: [Asterisk-video] mp4save(): different audio and
> > video lengths
> > >
> > > Hi, guys,
> > >
> > > I don't know if that's a known fact, I have discovered
> > that, at least
> > > in my systems, mp4save() creates mp4files with different
> lengths in
> > > audio and video tracks (see examples at the end). This
> > means that the
> > > audio track in those files is almost useless, as it is
> > totally out of
> > > synch with the video ...
> > >
> > > I think this has something to do with the framesPerSample
> > property in
> > > mpeg4ip package, but after messing a bit with the
> > > MP4 library I have surrended without fixing it. Have
> > somebody tested
> > > or fixed this before?
> > >
> > > Can you check by yourself if this behaviour also happens in your
> > > system?.
> > >
> > > Thanks,
> > > JM
> > >
> > >
> > > ~$ mp4info /tmp/*.mp4
> > > mp4info version 1.5.0.1
> > > /tmp/save.clock.1.mp4:
> > > Track Type Info
> > > 1 audio AMR, 6.780 secs, 12 kbps, 8000 Hz
> > > 2 hint Payload AMR for track 1
> > > 3 video H.263, 9.171 secs, 47 kbps, 176x144 @ 8.614110 fps
> > > 4 hint Payload H263-1998 for track 3
> > > /tmp/save.clock.2.mp4:
> > > Track Type Info
> > > 1 audio AMR, 52.460 secs, 12 kbps, 8000 Hz
> > > 2 hint Payload AMR for track 1
> > > 3 video H.263, 61.413 secs, 45 kbps, 176x144 @
> 9.069741 fps
> > > 4 hint Payload H263-1998 for track 3
> > > /tmp/save.clock.3.mp4:
> > > Track Type Info
> > > 1 audio AMR, 2.900 secs, 10 kbps, 8000 Hz
> > > 2 hint Payload AMR for track 1
> > > 3 video H.263, 6.649 secs, 49 kbps, 176x144 @ 8.572718 fps
> > > 4 hint Payload H263-1998 for track 3
> > > ~$
> > >
> > >
> > >
> > > _______________________________________________
> > > --Bandwidth and Colocation Provided by
> http://www.api-digital.com--
> > >
> > > asterisk-video mailing list
> > > To UNSUBSCRIBE or update options visit:
> > > http://lists.digium.com/mailman/listinfo/asterisk-video
> > >
> >
> >
> > _______________________________________________
> > --Bandwidth and Colocation Provided by http://www.api-digital.com--
> >
> > asterisk-video mailing list
> > To UNSUBSCRIBE or update options visit:
> > http://lists.digium.com/mailman/listinfo/asterisk-video
> >
> >
> > _______________________________________________
> > --Bandwidth and Colocation Provided by http://www.api-digital.com--
> >
> > asterisk-video mailing list
> > To UNSUBSCRIBE or update options visit:
> > http://lists.digium.com/mailman/listinfo/asterisk-video
> >
> >
> >
> > _______________________________________________
> > --Bandwidth and Colocation Provided by http://www.api-digital.com--
> >
> > asterisk-video mailing list
> > To UNSUBSCRIBE or update options visit:
> > http://lists.digium.com/mailman/listinfo/asterisk-video
> >
>
>
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
>
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
> http://lists.digium.com/mailman/listinfo/asterisk-video
>
>
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
>
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
> http://lists.digium.com/mailman/listinfo/asterisk-video
>
More information about the asterisk-video
mailing list