[Asterisk-video] Videomixing in MeetMe

[Lorenzo Miniero]

Hi Klaus,

I'm glad you got the mixer working!
My answers to your comments are inline.


> HI Lorenzo  - this fix works. Thus, I got mixed video. Below some comments:
> 
> - I have lots of troubles with eyebeam when video is activated. System 
> and eyebeam need lots of CPU resource and I can't do anything else on my PC.


Is it always so with eyebeam+video (with any other video call you have with
anyone), or only when eyebeam receives the frames from the mixer?


> 
> - ffmpeg reports some compilation problems when started - do you know 
> how to solve this?
> 
> Compiler did not align stack variables. Libavcodec has been miscompiled
> and may be very slow or crash. This is not a bug in libavcodec,
> but in the compiler. Do not report crashes to FFmpeg developers.


I frankly don't know what may cause this. You could try a make clean and
reconfigure/recompile/reinstall ffmpeg to see it gets fixed this way.


> 
> - There are lots of ffmpeg error messages during mixing:
> 
> -- this one happens always - even if there is only one, deactiated peer 
> and the logo is sent out:
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> ....


Yes, I noticed this too. I think it's related to rate control and to the fact
that the logo is a fixed frame (i.e. no motion), but I still couldn't
understand how to handle this.


> 
> -- this one happens very often during mixing
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
> [h263 @ 0xb7de8ca0]Error at MB: 164
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
> [h263 @ 0xb7de8ca0]Error at MB: 328
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
> [h263 @ 0xb7de8ca0]Error at MB: 492
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
> [h263 @ 0xb7de8ca0]Error at MB: 656
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
> [h263 @ 0xb7de8ca0]Error at MB: 820
> [h263 @ 0xb7de8ca0]I cbpy damaged at 0 24
> [h263 @ 0xb7de8ca0]Error at MB: 984
> [h263 @ 0xb7de8ca0]concealing 609 DC, 609 AC, 609 MV errors
> [h263 @ 0xb7de8ca0]Bad picture start code
> [h263 @ 0xb7de8ca0]header damaged
> ...
> [h263 @ 0xb7de8ca0]concealing 707 DC, 707 AC, 707 MV errors
> [h263 @ 0xb7de8ca0]Bad picture start code
> [h263 @ 0xb7de8ca0]header damaged
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
> [h263 @ 0xb7de8ca0]Error at MB: 164
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
> [h263 @ 0xb7de8ca0]Error at MB: 328
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
> [h263 @ 0xb7de8ca0]Error at MB: 492
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
> [h263 @ 0xb7de8ca0]Error at MB: 656
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
> [h263 @ 0xb7de8ca0]Error at MB: 820
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 24
> [h263 @ 0xb7de8ca0]Error at MB: 984
> ...


This could be related to two aspects:

        * the H.263 codec used by eyebeam could be not-100%-compliant with the
ffmpeg H.263 codec (and this is something we'd have to live with);
        * there might be problems with the H.263 payload header and how I handle
it.

I'm still investigating the H.263 payload header issue, also considering that
there are three different modes for it. In fact, different settings give
different results according to the client (for example, a setting that works
fine with our client gives strange behaviour with Wengophone, while settings
that work fine with Wengophone don't work at all with our client, and the same
applies with some other clients we tested). I'll try eyebeam myself as soon as
I can to see if I can sort it out. Of course, if you have advices for this, let
me know!


> 
> - I tested with 2 peers, thus the video was split into two parts 
> (left/right). One video source was shrunk (as it should be), but the 
> other source was enlarged (zoomed in): I found out that the enlarged 
> video had different settings in xlite. After setting bandwidth in xlite 
> from "Fast Cable ..." to "DSl, Cable..." it worked fine.


Considering how the layout for two sources is codec in the mixer, both the
videos should be shrunk (in all other layouts, sources are almost always simply
resized). The fact that changing bandwidth gave the expected result might imply
a fps-related issue: in fact, the mixer currently uses a fixed outgoing fps, so
very different in-fps and out-fps might explain this...


> 
> - If the videomixer is restarted, Asterisk fails to reconnect to the 
> videomixer.


Yes, the integration is very rough at the moment, since all is work-in-progress.
I focused on the mixing aspects, and still didn't take care of reconnections
and so on. If Asterisk fails at connecting to the videomixer at boot, or the
connection gets lost, there's no way to recontact the videomixer, and a restart
is needed. I'll take care of this in the next release.


> 
> - the videomixer does not close all UDP sockets when a conference is 
> terminated.


Same as above, not all resources, including connections and RTP channels, are
appropriately freed when the mixer shutdowns. I'll add this in the next
release.


> 
> Finally I have some questions about the architecture of the videomixer. 
> If I understand it correctly, the Meetme application receives the video 
> frames from Asterisk core (which received it from chan_sip) and forwards 
> it to the videomixer via UDP. Videomixer receives the video, mixes it, 
> and sends the streams back to Meetme. Meetme sends video to the SIP clients.
> 
> Why is ortp needed? Is the communication between videomixer and MeetMe 
> based on RTP and Meetme acts as bridge for the RTP video packets?


Exactly, Asterisk acts as an RTP bridge for media streams. Even if the ideal
case would be a direct media flow between users and the mixer, having Asterisk
in the video path is unavoidable at the moment. In fact, such a feature would
require appropriate renegotiation, which:

      1. can't be done in a transparent way for whatever channel driver (you can
only bridge channels);
      2. can't be done hacking a single channel driver as chan_sip either,
considering how chan_sip is conceived at the moment; in fact, redirecting the
video flow would require a media-specific c-line for video, a granularity that
is not possible in the current chan_sip because of the fact that the SDP
processing still is not media-specific at all; a CANREINVITE would not be the
solution since it would involve all the media in the SDP (so audio too).


> 
> regards
> klaus
> 


Thanks a lot for your precious feedback, it's much appreciated!
If you have any other question and/or advices/suggestions/criticisms about this
work, I'll be glad to hear them.

Regards,
Lorenzo

-- 
Lorenzo Miniero, Junior Researcher
Dipartimento di Informatica e Sistemistica
Università degli Studi di Napoli "Federico II"
Via Claudio 21 -- 80125 Napoli (Italy)
Phone: +390817683821 - Fax: +390817683816
Email: lorenzo.miniero at unina.it


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.