[Asterisk-video] mp4save(): different audio and video lengths

Tue May 13 07:33:35 CDT 2008

Hi Sergio,

I've some comments to your mail:

Sergio Garcia Murillo wrote:
> Hi Sven,
> 
> As I said I think that the patch is correct and I hope to apply it soon.
> I'd like to get some feedback first from the people that have reported 
> the problem so we all now the impact of the solution and if it solves 
> defenetivelly the problem.
> 

Agree.

> About AL2,3 an SN.. you made me have to investigate my own code and 
> recover my h223 spec from the bottom of one of my drawers.. jeje..

Good place for specs ;-)

> I finished that part of the code a year ago, so IIRC I used AL2 with SN 
> for both audio and video.
> 

After looking to the source code of libh324, I guess the code is able to use 
SN's. I think it would be not so difficult to extend the code to calculate time 
stamps from SN's. But, if not every phone supports this SN feature, it would be 
senseless to implement such one. I don't know the numbers of phone that support 
SN's.

> I'll have to take a deeper look and see how did I solved the problem of 
> dropped packets in app_mp4 as this would not only apply to h234m, you 
> could have the same issue in RTP. The solution is to specify the 
> timestamp of the amr packet in the asterisk sample strutcture, this 
> would probably mean to store the received amr frame until we receive the 
> next to know how much delay it's between both so we behave the same as 
> it's been done in RTP. Then app_mp4 would calculate the audio frame 
> timestamp according to that and solve the issue. I'll check how much of 
> this is actually implemented.
>

If I understand you right, the solution would be to record the time when the 
audio/video frame arrives at the libh324 and calculate the time between the 
frames. So we can detect lost frames.
This would only work on bandwidth and delay constant channels, h324m is a 
circuit switched technique - I mean this could work.

> About aduo/video different lengths, it's normal.. first audio is sent at 
> 20 ms intervals and video is sent at 1/fps (for 10fps is each 100ms). 
> Second  audio is shorted and arrives first than video, so you could have 
> up two frames of difference in length (200ms). If there is more 
> difference will have to take a deeper look at it.
> 

Right. :-)

> Best regards
> Sergio
> 
> 
> Sven Brandau escribió:
>> Hi Sergio, Hi @All,
>>
>> I think the solution that introduced by me works fine on none error 
>> prone channels.
>>
>> In case of errors or lost frames the patch will not have any impact. I 
>> understand that the adaption layer 2 for audio (AL2) and layer 3 for 
>> video (AL3) of H.324M can be used with optional sequence numbers (SN). 
>> By using sequence numbers, the receiver can detect lost frames.
>>
>> But the use of SN is optional and depends on the implementation of the 
>> H.324M stack on the mobile phone. If the phone doesn't support SN's we 
>> can't retrieve lost frames.
>>
>> In terms of the problem here with desynchronized audio/video or 
>> different lengths of audio/video, I guess we could solve it only on 
>> phones that supports sequence numbers.
>>
>> Am I right?
>>
>> Best regards,
>> Sven
>>
>>
>> Sergio Garcia Murillo wrote:
>>> Yes, please, if anyone test it, send the results to the list so we 
>>> can commit the patch to the repository.
>>> A priori I see it quite clea, so I think it won't do any harm at all 
>>> (another issue is if it solves the problem :)
>>>
>>> BR
>>> Sergio
>>>
>>> Sven Brandau escribió:
>>>> Hi @All,
>>>>
>>>> recording to the questions, I've send you my patch.
>>>>
>>>> Please note that the patch is NOT really tested - please try it
>>>> and send me your results.
>>>>
>>>> Patch it with (R213):
>>>> patch -i app_h324m.c.patch
>>>>
>>>> Regards,
>>>> Sven
>>>>
>>>>
>>>>
>>>>
>>>> Sergio Garcia Murillo wrote:
>>>>> Hi Sven
>>>>>
>>>>> Could you send me a patch with oyour solution?
>>>>>
>>>>> BR
>>>>> Sergio
>>>>>
>>>>> ----- Original Message -----
>>>>> From: Sven Brandau [mailto:brandau at gmx.de]
>>>>> To: asterisk-video at lists.digium.com
>>>>> Sent: Mon, 05 May 2008 15:30:33 +0200
>>>>> Subject: Re: [Asterisk-video] mp4save(): different audio and video 
>>>>> lengths
>>>>>
>>>>> Hi @All,
>>>>>
>>>>> I'm working on a solution.
>>>>>
>>>>> I've found that some mobile phones (Moto K3) sending out AMR No-Data
>>>>> packets, instead of comfort noise frames (AMR-SID). So I replaced the
>>>>> No-Data packets with the last received AMR-SID frame (file 
>>>>> app_h324m.c /
>>>>> function create_ast_frame). This solution worked fine for me.
>>>>>
>>>>> The only thing is, that the recording for audio and video starts on a
>>>>> different time - audio and video is already not synchronized, but only
>>>>> a few milliseconds. It depends on the mp4save option /V - waiting 
>>>>> for the
>>>>> first I-frame.
>>>>>
>>>>> I'm already working on this issue and will announce my solution 
>>>>> here on the
>>>>> mailing list.
>>>>>
>>>>> Regards,
>>>>> Sven
>>>>>
>>>>> Low Yu Siang wrote:
>>>>>> Hi,
>>>>>>
>>>>>> May I know if anyone has solved the recording
>>>>>> audio/video length problem?
>>>>>>
>>>>>> Tech from i6net wrote :
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am going to work on it.
>>>>>>>
>>>>>>> But for the moment, I am qualifying the video transcoder and the 
>>>>>>> h324m module.
>>>>>>> The Gateway (SIP -> 3G or 3G -> SIP) configuration
>>>>>> locks after some secondes or minutes if it use the
>>>>>> transcoder.
>>>>>>> 2 ways for the recording :
>>>>>>> - Replace the silence by generating the AMR datas
>>>>>> with a audio silence.
>>>>>>> - Check if we can timestamps correctly the AMR
>>>>>> packets in the MP4/3GP hinted file.
>>>>>>
>>>
> 

-- 
----------------------------------------------------------------------
Dipl. Ing. Sven Brandau                     Software Systems Architect

Tel:    +49-30-70033914                        Freelancer & Consultant
Fax:    +49-30-75633961
Mobile: +49-173-9960100
WWW:    www.brandau.biz
Email:  brandau at gmx.de                              Anhalter Strasse 7
         info at brandau.biz                                D-10963 Berlin
----------------------------------------------------------------------