[Asterisk-video] lip sync issues with mp4_play

[Dan Julius]

Thanks everyone for all the replies.

Does h263+ not have PTS/DTS information similar to MPEG which tell the
mobile phone when to decode and display each frame?
I figure that if such information would exist in the stream, it could solve
the A/V sync issues.

Dan

On Wed, Nov 12, 2008 at 12:44 AM, Sergio Garcia Murillo <
sergio.garcia at fontventa.com> wrote:

>  Hi Dan,
>
> Lip sync?? jeje.. good one.. ;)
>
> Let's start with the bad news there is no audio/video sync mechanism in
> asterisk (as we comment in a previous mail), the good news are that if you
> have two synced media streams input in asterisk, the two media streams
> outputs are going to be synced.
>
> So let's go for the source, app_mp4. As Emmanuel says in another mail, the
> hint track marks the timing for the rtp packets to be sent, so if the source
> i synced, app_mp4 will play it synced.
> The bad news is that mp4 files are prepared for streaming, not for been
> transmitted on a fixed rate channel and played on a live, non-buffered
> player. So, what's the problem?
> First, app_h324m does nothing with the sync, it just receives the packets
> from asterisk, extract the media and send it to the h324m library, so far so
> good.
>
> The h324m library has a small jitter buffer, and it does not start sending
> until it has a minimum of data on it (3 packets if I remember correctly), so
> it introduces a small delay. Why? because if audio data is not sent with an
> almost perfect timing the mobile phone will suffer an audio under run and
> the audio will be crappy. But 3 packets are 60ms, so not much to worry.
> When the jitter has enough data it starts sending audio at a fixed rate,
> and audio data will have preference over video, so the audio will be
> constantly played, no variable delays introduced.
>
> Video is another issue, the biggest problem are the encoders. Most of them,
> ffmpeg at least, are not constant bitrate encoders. So they will produce a
> very big IFrame followed by small PFrames. What's the problem with that? The
> ISDN can't send faster, so a variable delay will be introduced on video
> depending on the size of the frame.
>
> What happens then? Did you remember that we said that audio and video were
> RECEIVED synced on the app_h324m application? So when you sent the audio and
> video frame to the h324m library they are still synced, and even the h324m
> library starts sending it almost synced. But the amr arrives much more
> faster than the video, that depending on the size could take almost a second
> to be sent depending on the video bitrate.
>
> What do mobile phones do? Usually a best effort approach, as soon as they
> receive audio, they play it. As soon as they receive the a video frame they
> decode and display it. That is the reason why sometimes a video plays faster
> than it should be for a while (several frames have been enqueued, been the
> latest very small, and they are received by the mobile phone almost at the
> same time).
>
> What can be done? not much, buffering will introduce an unacceptable delay
> in a person to person communication, so the only good solution is to be ale
> to have a CBR h263 encoder that outputs always the same amount of data for
> each frame. That way a fixed delay will be introduced in video and by
> adjusting the jitter the synchronization we could be almost perfect.
>
> Best regards!
> Sergio
>
> Dan Julius escribió:
>
>  Hi,
>
> Is there a good reference to better understand how audio/video
> synchronization is handled in h324m?
>
> I'm using mp4_play to play a file I created following the mp4 mini
> tutorial. I combined an h263 track with a g711 track,
> I've noticed that by modifying the bitrate of the video channel I can get
> different results, but all of them have lip-sync issues.
>
> where is the av sync supposed to be handled, in the mp4 file, by libh324m,
> or the 3g handset?
>
> Thanks for the advice,
> Dan
>
>
> ------------------------------
>
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
>
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-video
>
>
>
> _______________________________________________
> --Bandwidth and Colocation Provided by http://www.api-digital.com--
>
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-video
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.digium.com/pipermail/asterisk-video/attachments/20081112/470194c4/attachment.htm