[Asterisk-video] Re: [Iaxclient-devel] Video codec negotiation in IAX

[Cristian Draghici]

On 10/23/06, Mihai Balea <mihai at hates.ms> wrote:
> Hello all,
>
> As you may already know, I am working on adding video support to
> iaxclient.
>
> It appears however that IAX's support for video codec negotiation is
> lacking.  I am no IAX expert, but from what I can see, there are to
> ways of doing codec negotiation in IAX:
> - use the FORMAT and CAPABILITY IEs.  This allows you to specify one
> preferred codec and a list of acceptable codecs. This is the way
> iaxclient does codec negotiation at this time.
> - use the CODEC PREFS IE.  This allows you to specify a list of
> acceptable codecs, in order of preference.
>
> The problem with video is that you need to negotiate two codecs at
> the same time (one for video and one for audio). Unfortunately, none
> of the above mentioned methods are suitable for this.  The way I see
> it, there are several ways of addressing this:
> 1. use a VIDEO FORMAT IE, in addition to FORMAT and CAPABILITY.  The
> CAPABILITY IE will contain acceptable video codecs as well as audio.
> 2. use a VIDEO CODEC PREFS IE in addition to the current CODEC PREFS
> IE.  This will more flexible since it allows for video codec
> preference order.
> 3. use the current FORMAT and CAPABILITY and change the semantics of
> FORMAT to allow a video codec as well as an audio codec. This might
> be easiest to implement.
>
> As a long term solution, I'm inclined to go with #2 since it provides
> the maximum amount of flexibility.  However I would like to hear any
> suggestions you might have regarding this issue...
>
> Thanks in advance,
> Mihai
>

I'm pasting a message I've written a while ago on the iax client list
- i think it's very much in tune to what's being discussed.
Background on this discussion is trying to get an video echo call
iaxclient <-> Asterisk to work.

- when testing a SIP echo video call (which works audio and video) I
see that the SIP client starts off with audio only
- at some point in the future the client adds video frames to the mix
(user says - yes, start video)

This does not look like it could be supported by the current version
of iaxclient.
iaxclient tries to do the negociation at call set-up time (i.e. when
IAX NEW and ACCEPT frames are sent and received) and it doesn't change
it in mid call.

I am wondering if Asterisk is not doing a codec "renegociation" in mid
call and we're trying to mix apples and oranges by trying to persuade
chan_iax into doing something it can't...

I'm referring to this fragment from chan_iax2.c (in function
socket_read, asterisk version 1.2.something):

if (f.frametype == AST_FRAME_VIDEO) {
                       if (f.subclass != iaxs[fr->callno]->videoformat) {
                               ast_log(LOG_DEBUG, "Ooh, video format
changed to %d\n", f.subclass & ~0x1);
                               iaxs[fr->callno]->videoformat =
f.subclass & ~0x1;
                       }
               }
I don't see any logic in this code as long as chan_iax would not
support mid-call codec changes.

from my point of view, the ideal iaxclient would
- advertise accepting video frames even if a camera is not present
- advertise accepting audio frames even if microphone is not present
(think monitor, i.e. read voice mail where you only input DTMF digits)
- if a call from the call list has a ulaw codec set and a (i imagine
big frame) video frame arrives from the iaxclient remote peer, i think
the local peer should try and match the codec for the received video
frame and if it succeeds mask the codec capabilities with this codec
(this would allow iax peers to turn cameras off and on whilst in mid
call).

--
Cristi