[Asterisk-video] RE: AVTF - What's going on?
Shidan
shidan at gmail.com
Tue May 16 16:45:32 MST 2006
Sorry about my off topic post everyone, I meant it just for the
original poster and sent it to the list by accident.
----
Shidan
On 5/16/06, Shidan <shidan at gmail.com> wrote:
> >We have a number of customers that would like to use our video phones in
> > Asterisk but have been missing a way of doing the sort of things that
> > you would normally expect of a phone but with video tacked on.
>
>
> To be honest with you, with that huge touch screen of yours, what
> would make aupix a much more asterisk friendly phone is if it
> supported easy application development, the phones are nice but if you
> embedded a web browser, possibly the flash player or a nice simple and
> free (as in cash) java api to an asterisk manager proxy it would take
> advantage of your console for some amazing asterisk based
> applications. For sure its more important to business now than video
> support.
>
> If you don't have time for something like that contact me off list,
> thats something my company would enjoy collaborating with you on.
>
> ---
> Shidan
>
> On 5/16/06, John Martin <John.Martin at aupix.com> wrote:
> > Hi All,
> > I know you're all a long way into video for Asterisk in some way or
> > other but I thought I'd outline what we're trying to do, what we've got
> > going and what we're still playing with. If everyone is prepared to show
> > and tell then perhaps we can figure out where we should be going and how
> > to get there... sorry if this ends up being a rather long winded mail.
> >
> > So what are AuPix trying to achieve?
> >
> > We have a number of customers that would like to use our video phones in
> > Asterisk but have been missing a way of doing the sort of things that
> > you would normally expect of a phone but with video tacked on. Our
> > phones support SIP and H.323 so that's what we've majored on. We have
> > customers doing regular telephony stuff and ones doing contact center
> > things, so we needed to address all of this in Asterisk.
> >
> > So what'a working?:
> > Well we've been adding video things to SIP and H.323 since Asterisk
> > 1.0.7. Initially it was simple stuff like assuming H.263, CIF, MPI=1,
> > that sort of thing, and building from there. In various Asterisk
> > releases we've found things get broken, (i.e since around 1.2.something
> > we found that chan->nativeformats got a bit more intelligent and so
> > broke a number of channel cap exchanges) so we fix and patch as we go.
> >
> > As of today the following have been changed, tweaked, hacked or
> > generally man handled by us:
> >
> > Chan_sip:
> > - maxcallbitrate: a basic way of telling an endpoint what bitrate it
> > should use (our videophones can do 2Mbps and so assuming such a rate for
> > a DSL connection was never going to make for great call quality, see
> > RTCP below). This should be superseded by a better capability mechanism.
> >
> > - per peer videosupport: some carriers really barf when you send them a
> > video cap so we needed a way of allowing some endpoints to have video
> > support and some not (i.e. internal extensions have video but some
> > external trunks only audio).
> > - per peer INFO or RTCP fast update requests: again some switches don't
> > appreciate SIP info messages with XML in them as the Fast Update Request
> > mechanism, so we added a per peer support way of selecting between
> > sending SIP INFO with XML or by doing RTCP 192 (which is an old H.261
> > way of doing FUR) again see RTCP below.
> > - Process SDP: we have a simple way of extracting video fmtp lines from
> > SDP. It's a bit ugly and has little awareness of SDP scope at the moment
> > but it does extract fmtp for H.261, H.263 and H.264 into a common
> > ast_capabilities (see channel.c below).
> > - Build SDP: having extracted the fmtp lines and put them in a struct we
> > pass the struct around to the called endpoints (changes to the bridging
> > functions in channel.c) to limit their cap. We do a min() on confcap,
> > peercap and chancap to determine a maximum jointcap that an endpoint
> > will be allowed to generate once it gets an ack with SDP - don't get us
> > going on early media responses because it breaks some switches that
> > should know better.
> > - Video capability in SIP.conf: we use the h.261/3 fmtp definitions to
> > specify video capabilities in the conf file. i.e. you can specify lines
> > like:
> > H261=CIF=1 QCIF=2 CIF16=4 MaxBR=2024 framerate=25
> > H263=CIF=1 QCIF=2 CIF16=4 QVGA MaxBR=2024 A/B/C
> > Etc.
> > I know you wouldn't normally do an MPI and framerate but we support it.
> > We also allow for incorrectly formatted fmtp like some vendors are want
> > to do.
> >
> > Chan_h323:
> > - Well we added video for a start, sounds like others have been having
> > similar fun too :-) . We also added the same maxcallbitrate and per peer
> > videosupport as for SIP.
> > - H.261, H.263 and nearly H.264 (I've got H.264 doing all the cap
> > exchanges and getting the media channels running, I'm just patching in
> > all the dynamic RTP stuff needed by Asterisk, h.323 is doing dynamicRTP
> > - I just need to let Asterisk know about it. If anyone else has got
> > H.264 going I'd love to see their approach).
> > - There's also the beginnings of the same fmtp like conf options for
> > H.323 in the conf file.
> > - Fast update requests received and transmitted.
> > - Fixes in H.323 for various Tandberg endpoints that don't like the way
> > chan_h323 used to do port reflection (this is a pain for NAT support but
> > most of our customers with H.323 seem to have stuck with DMZ or public
> > IP address as the work around for NAT on H.323).
> > - Video codecs are now broken out into their own files: ast_h26x.cpp and
> > the makefile fixed to allow the links to *.cxx to work.
> > - Logger support is also broken out into it's own file.
> > - We're currently using RFC2190 support for H.263 but I think we'll do
> > our own capability for that in the next few days.
> >
> > Chan_local:
> > - it just needs a .write_video in the channel tech to get this working
> > (I think that was all, I'd have to do a diff to see if we fixed anything
> > else).
> >
> > Chan_agent:
> > - similar to chan_local (.write_video) but I think there may have been a
> > bit more required.
> >
> > App_echo:
> > - simply added a ast_indicate at the front of the app so that if you do
> > a playback or IVR before hitting app_echo then you don't get any
> > breakthrough on some endpoints.
> >
> > Func_stringtoip:
> > - this is one of our own creations that allows a user to be registered
> > with chan_sip, or with a gatekeeper, and enter some arbitrary IP address
> > like 10*1*1*1, or 010001001001, to make a call to 10.1.1.1 (I know that
> > should be an internal IP address, replace with the public IP address of
> > your choice). This is useful when you're supporting local extensions
> > that want to talk to, for instance, lots of D-Link i2eyes that are all
> > essentially sitting in the DMZ of a CPE router and not registered. I
> > guess this could have been done in a set of dialplan apps but I'd rather
> > do in in C.
> >
> > RTP.c:
> > Hmmmm.... lots of very good work was done on this before we shoved our
> > nose into it. We only added a few tweaks to RTP.c for RTCP but I think
> > it works now when video is involved too.
> > - RTP.c was seg faulting when the channel drivers shut down the RTP
> > layer but the RTCP timers were still going - no prizes for guessing that
> > the timer routine barfed when it tried to access the non-existent rtp
> > structs.
> > - Tidied up the parsing of the RTCP compound packets so that it worked.
> > - Generate compound RTCP packets (you should always send at least two
> > packets so we added a dummy SDES - it should really be filled in).
> > - FUR now generated from the channel drivers if they want it. Our
> > chan_sip decides whether to do it this way or with sip info. Some sort
> > of auto refelction should be possible but it may not always be reliable.
> > - FUR reported back to the channel drivers, thanks to Neil Stratford for
> > pointing out a blindingly obvious way of doing it that us mere mortals
> > had completely missed.
> > - Fixed up some report errors on seqno wraparound etc.
> > - A bit more of a troublesome problem for us has been seqno
> > resequencing. Asterisk puts it's own sequence numbers into outbound RTP
> > packets and ignores the inbound ones. This is generally fine for audio
> > but we found that the lumpy nature of high bandwidth video calls could
> > produce packet races (packets get swapped in transit to Asterisk)
> > through the internet but Ast would renumber the outbound packets. This,
> > of course, means that when the packet hits the target endpoint (having
> > passed through Asterisk) it's actually out of sequence but the endpoint
> > has no way of knowing that because all the seqno's are sequential
> > (unless there's been an Ast to target swap aswell!). We've added support
> > for seqno being added to ast_frame so that the RTP layer can use the
> > inbound seqno if it's available. This gets complex with IVR and other
> > file playback functions. Our solution can also cause large seqno jumps
> > when going from file playback to live video, which some endpoints will
> > crash on... the RFC says they shouldn't ;-) This fix isn't in Olle's
> > RTCP branch as I thought it might be a little too dangerous.
> > - RTCP report frequency in conf: The rtp.conf now supports an
> > rtcpinterval=x ms setting to set how often rtcp reports are generated.
> > The default 5000msecs isn't really enough to be handling video rate
> > control. I think the RFC recommends that 5% of packets should be RTCP
> > reports - that seemed a bit high to us, but sending one every 5 seconds
> > when you've sent over 3000 video packets in that time (0.03%) seemed too
> > low too :-)
> > - DTMF tones (not really a video one this, but I'm on a roll so bear
> > with me): I think there might be something very wrong with the DTMF tone
> > generation in Asterisk. We get a lot of complaints about it and have
> > tweaked what's there to improve things. We're waiting for Kevins's
> > variable length DTMF support 'cos I'm sure that'll be much better.
> >
> > Channel.c/h
> > - We've come up with a struct to store all the new video caps in. It
> > looks like this but I'm sure there'll be lots of views on the way it
> > should be done and propagated around Asterisk (the codecnegociation
> > branch would maybe be a good start)
> >
> > #define AST_VIDEO_SQCIF (1)
> > #define AST_VIDEO_QSIF (2)
> > #define AST_VIDEO_QCIF (3)
> > #define AST_VIDEO_QVGA (4)
> > #define AST_VIDEO_SIF (5)
> > #define AST_VIDEO_CIF (6)
> > #define AST_VIDEO_VGA (7)
> > #define AST_VIDEO_4CIF (8)
> > #define AST_VIDEO_SVGA (9)
> > #define AST_VIDEO_XGA (10)
> > #define AST_VIDEO_16CIF (11)
> >
> > #define AST_VIDEO_LEVEL_BASELINE (1 << 0)
> > #define AST_VIDEO_LEVEL_MAIN (1 << 1)
> > #define AST_VIDEO_LEVEL_EXTENDED (1 << 2)
> >
> > #define MAX_VIDEO_SIZES 11
> >
> > /* The following array defines the video sizes we know about */
> > struct videoSize{
> > char* type;
> > unsigned int num_mbs;
> > unsigned int maxframterate;
> > };
> >
> > static struct videoSize videoSizes[MAX_VIDEO_SIZES] = {
> > {"SQCIF", 48, 0},
> > {"QSIF", 75, 0},
> > {"QCIF", 99, 0},
> > {"QVGA", 300, 0},
> > {"SIF", 300, 0},
> > {"CIF", 396, 0},
> > {"VGA", 1200, 0},
> > {"CIF4", 1584, 0},
> > {"SVGA", 1875, 0},
> > {"XGA", 2304, 0},
> > {"CIF16", 6336, 0},
> > };
> >
> > /*! \brief ast_video_cap: Struct holding video capabilities */ struct
> > ast_h2613_video_cap {
> > int valid;
> > /*!< 1 = This codec is specified, 0 = not specified */
> > unsigned int annexes;
> > /*!< Video annexes - stored as bitfield 2^(annex-ascii(A))*/
> > unsigned int maxframerates[MAX_VIDEO_SIZES]; /*!< Maximum
> > framerate (framerate = 30/mpi) */
> > unsigned int maxbr;
> > /*!< Maximum supported bitrate for this capability */
> > };
> >
> > /*! \brief ast_h264_video_cap: Struct holding video capabilities */
> > struct ast_h264_video_cap {
> > unsigned int flags; /*!< Video flags
> > */
> > unsigned int level; /*!< Profile
> > level */
> > unsigned int maxmbps; /*!< Maximum number of
> > macroblocks per second */
> > unsigned int minfs; /*!< Minimum
> > frame size in macroblocks */
> > unsigned int maxfs; /*!< Maximum
> > frame size in macroblocks */
> > unsigned int maxcpb; /*!< Maximum coded
> > picture buffer size in units of 1000 bits (1200 NAL HRD)*/
> > unsigned int maxdpb; /*!< Maximum decoded
> > picture buffer size in units of 1024 bytes */
> > unsigned int packet_mode; /*!< Packetisation mode */
> > unsigned int maxbr; /*!< Maximum
> > supported bitrate for this capability */
> > };
> >
> > /*! \brief ast_video_capabilities: Struct holding all video capabilities
> > */ struct ast_video_capabilities {
> > unsigned int maxbitrate; /*!< Max video bitrate */
> > struct ast_h2613_video_cap h261; /*!< H261 video capabilities
> > */
> > struct ast_h2613_video_cap h263; /*!< H263 video capabilities
> > */
> > struct ast_h264_video_cap h264; /*!< H264 video capabilities
> > */ };
> >
> > /*! \brief ast_video_capabilities: Struct holding all video capabilities
> > */ struct ast_capabilities {
> > int maxcallbitrate;
> > /*!< Max allowed bitrate for this channel */
> > struct ast_video_capabilities videocaps; /*!< video
> > capabilities */ };
> >
> > I think that's enough for now... if I've still got any of you with me
> > then perhaps the future of what we're up to can come another time....
> >
> > John
> >
> >
> > John Martin
> > John.Martin at AuPix.com
> > http://www.AuPix.com
> >
> > -----Original Message-----
> > From: Olle E Johansson [mailto:registry at webway.se]
> > Sent: 16 May 2006 10:13
> > To: asterisk-video at edvina.net
> > Subject: AVTF - What's going on?
> >
> > This is stuff going on that I'm aware of:
> >
> > * I am working on fixes for SIP/SDP in the sdpcleanup branch.
> > Grandstream and Foniris generously
> > donated video phones for me to test with. I am implementing some
> > checks if we really have
> > video on the incoming call before we offer it on the outbound
> > call. Seems stupid to offer
> > video without a stream. If we can fix this stuff and test it
> > quickly, I can get it into 1.4.
> >
> > * John Martin contributed some patches for FMTP support that is
> > needed. Will open a branch
> > for this hopefully this week. This is too late for 1.4, but will
> > have to be part of 1.6. It will require
> > some architectural changes, and we're beyond freeze for that.
> >
> > * John Martin is also doing work on H.323 and video
> >
> > What else is going on codewise or should be added to the wish list?
> >
> > /Olle
> >
> >
> > _______________________________________________
> > --Bandwidth and Colocation provided by Easynews.com --
> >
> > asterisk-video mailing list
> > To UNSUBSCRIBE or update options visit:
> > http://lists.digium.com/mailman/listinfo/asterisk-video
> >
>
More information about the asterisk-video
mailing list