[Asterisk-video] RE: AVTF - What's going on?
Shidan
shidan at gmail.com
Tue May 16 16:43:57 MST 2006
>We have a number of customers that would like to use our video phones in
> Asterisk but have been missing a way of doing the sort of things that
> you would normally expect of a phone but with video tacked on.
To be honest with you, with that huge touch screen of yours, what
would make aupix a much more asterisk friendly phone is if it
supported easy application development, the phones are nice but if you
embedded a web browser, possibly the flash player or a nice simple and
free (as in cash) java api to an asterisk manager proxy it would take
advantage of your console for some amazing asterisk based
applications. For sure its more important to business now than video
support.
If you don't have time for something like that contact me off list,
thats something my company would enjoy collaborating with you on.
---
Shidan
On 5/16/06, John Martin <John.Martin at aupix.com> wrote:
> Hi All,
> I know you're all a long way into video for Asterisk in some way or
> other but I thought I'd outline what we're trying to do, what we've got
> going and what we're still playing with. If everyone is prepared to show
> and tell then perhaps we can figure out where we should be going and how
> to get there... sorry if this ends up being a rather long winded mail.
>
> So what are AuPix trying to achieve?
>
> We have a number of customers that would like to use our video phones in
> Asterisk but have been missing a way of doing the sort of things that
> you would normally expect of a phone but with video tacked on. Our
> phones support SIP and H.323 so that's what we've majored on. We have
> customers doing regular telephony stuff and ones doing contact center
> things, so we needed to address all of this in Asterisk.
>
> So what'a working?:
> Well we've been adding video things to SIP and H.323 since Asterisk
> 1.0.7. Initially it was simple stuff like assuming H.263, CIF, MPI=1,
> that sort of thing, and building from there. In various Asterisk
> releases we've found things get broken, (i.e since around 1.2.something
> we found that chan->nativeformats got a bit more intelligent and so
> broke a number of channel cap exchanges) so we fix and patch as we go.
>
> As of today the following have been changed, tweaked, hacked or
> generally man handled by us:
>
> Chan_sip:
> - maxcallbitrate: a basic way of telling an endpoint what bitrate it
> should use (our videophones can do 2Mbps and so assuming such a rate for
> a DSL connection was never going to make for great call quality, see
> RTCP below). This should be superseded by a better capability mechanism.
>
> - per peer videosupport: some carriers really barf when you send them a
> video cap so we needed a way of allowing some endpoints to have video
> support and some not (i.e. internal extensions have video but some
> external trunks only audio).
> - per peer INFO or RTCP fast update requests: again some switches don't
> appreciate SIP info messages with XML in them as the Fast Update Request
> mechanism, so we added a per peer support way of selecting between
> sending SIP INFO with XML or by doing RTCP 192 (which is an old H.261
> way of doing FUR) again see RTCP below.
> - Process SDP: we have a simple way of extracting video fmtp lines from
> SDP. It's a bit ugly and has little awareness of SDP scope at the moment
> but it does extract fmtp for H.261, H.263 and H.264 into a common
> ast_capabilities (see channel.c below).
> - Build SDP: having extracted the fmtp lines and put them in a struct we
> pass the struct around to the called endpoints (changes to the bridging
> functions in channel.c) to limit their cap. We do a min() on confcap,
> peercap and chancap to determine a maximum jointcap that an endpoint
> will be allowed to generate once it gets an ack with SDP - don't get us
> going on early media responses because it breaks some switches that
> should know better.
> - Video capability in SIP.conf: we use the h.261/3 fmtp definitions to
> specify video capabilities in the conf file. i.e. you can specify lines
> like:
> H261=CIF=1 QCIF=2 CIF16=4 MaxBR=2024 framerate=25
> H263=CIF=1 QCIF=2 CIF16=4 QVGA MaxBR=2024 A/B/C
> Etc.
> I know you wouldn't normally do an MPI and framerate but we support it.
> We also allow for incorrectly formatted fmtp like some vendors are want
> to do.
>
> Chan_h323:
> - Well we added video for a start, sounds like others have been having
> similar fun too :-) . We also added the same maxcallbitrate and per peer
> videosupport as for SIP.
> - H.261, H.263 and nearly H.264 (I've got H.264 doing all the cap
> exchanges and getting the media channels running, I'm just patching in
> all the dynamic RTP stuff needed by Asterisk, h.323 is doing dynamicRTP
> - I just need to let Asterisk know about it. If anyone else has got
> H.264 going I'd love to see their approach).
> - There's also the beginnings of the same fmtp like conf options for
> H.323 in the conf file.
> - Fast update requests received and transmitted.
> - Fixes in H.323 for various Tandberg endpoints that don't like the way
> chan_h323 used to do port reflection (this is a pain for NAT support but
> most of our customers with H.323 seem to have stuck with DMZ or public
> IP address as the work around for NAT on H.323).
> - Video codecs are now broken out into their own files: ast_h26x.cpp and
> the makefile fixed to allow the links to *.cxx to work.
> - Logger support is also broken out into it's own file.
> - We're currently using RFC2190 support for H.263 but I think we'll do
> our own capability for that in the next few days.
>
> Chan_local:
> - it just needs a .write_video in the channel tech to get this working
> (I think that was all, I'd have to do a diff to see if we fixed anything
> else).
>
> Chan_agent:
> - similar to chan_local (.write_video) but I think there may have been a
> bit more required.
>
> App_echo:
> - simply added a ast_indicate at the front of the app so that if you do
> a playback or IVR before hitting app_echo then you don't get any
> breakthrough on some endpoints.
>
> Func_stringtoip:
> - this is one of our own creations that allows a user to be registered
> with chan_sip, or with a gatekeeper, and enter some arbitrary IP address
> like 10*1*1*1, or 010001001001, to make a call to 10.1.1.1 (I know that
> should be an internal IP address, replace with the public IP address of
> your choice). This is useful when you're supporting local extensions
> that want to talk to, for instance, lots of D-Link i2eyes that are all
> essentially sitting in the DMZ of a CPE router and not registered. I
> guess this could have been done in a set of dialplan apps but I'd rather
> do in in C.
>
> RTP.c:
> Hmmmm.... lots of very good work was done on this before we shoved our
> nose into it. We only added a few tweaks to RTP.c for RTCP but I think
> it works now when video is involved too.
> - RTP.c was seg faulting when the channel drivers shut down the RTP
> layer but the RTCP timers were still going - no prizes for guessing that
> the timer routine barfed when it tried to access the non-existent rtp
> structs.
> - Tidied up the parsing of the RTCP compound packets so that it worked.
> - Generate compound RTCP packets (you should always send at least two
> packets so we added a dummy SDES - it should really be filled in).
> - FUR now generated from the channel drivers if they want it. Our
> chan_sip decides whether to do it this way or with sip info. Some sort
> of auto refelction should be possible but it may not always be reliable.
> - FUR reported back to the channel drivers, thanks to Neil Stratford for
> pointing out a blindingly obvious way of doing it that us mere mortals
> had completely missed.
> - Fixed up some report errors on seqno wraparound etc.
> - A bit more of a troublesome problem for us has been seqno
> resequencing. Asterisk puts it's own sequence numbers into outbound RTP
> packets and ignores the inbound ones. This is generally fine for audio
> but we found that the lumpy nature of high bandwidth video calls could
> produce packet races (packets get swapped in transit to Asterisk)
> through the internet but Ast would renumber the outbound packets. This,
> of course, means that when the packet hits the target endpoint (having
> passed through Asterisk) it's actually out of sequence but the endpoint
> has no way of knowing that because all the seqno's are sequential
> (unless there's been an Ast to target swap aswell!). We've added support
> for seqno being added to ast_frame so that the RTP layer can use the
> inbound seqno if it's available. This gets complex with IVR and other
> file playback functions. Our solution can also cause large seqno jumps
> when going from file playback to live video, which some endpoints will
> crash on... the RFC says they shouldn't ;-) This fix isn't in Olle's
> RTCP branch as I thought it might be a little too dangerous.
> - RTCP report frequency in conf: The rtp.conf now supports an
> rtcpinterval=x ms setting to set how often rtcp reports are generated.
> The default 5000msecs isn't really enough to be handling video rate
> control. I think the RFC recommends that 5% of packets should be RTCP
> reports - that seemed a bit high to us, but sending one every 5 seconds
> when you've sent over 3000 video packets in that time (0.03%) seemed too
> low too :-)
> - DTMF tones (not really a video one this, but I'm on a roll so bear
> with me): I think there might be something very wrong with the DTMF tone
> generation in Asterisk. We get a lot of complaints about it and have
> tweaked what's there to improve things. We're waiting for Kevins's
> variable length DTMF support 'cos I'm sure that'll be much better.
>
> Channel.c/h
> - We've come up with a struct to store all the new video caps in. It
> looks like this but I'm sure there'll be lots of views on the way it
> should be done and propagated around Asterisk (the codecnegociation
> branch would maybe be a good start)
>
> #define AST_VIDEO_SQCIF (1)
> #define AST_VIDEO_QSIF (2)
> #define AST_VIDEO_QCIF (3)
> #define AST_VIDEO_QVGA (4)
> #define AST_VIDEO_SIF (5)
> #define AST_VIDEO_CIF (6)
> #define AST_VIDEO_VGA (7)
> #define AST_VIDEO_4CIF (8)
> #define AST_VIDEO_SVGA (9)
> #define AST_VIDEO_XGA (10)
> #define AST_VIDEO_16CIF (11)
>
> #define AST_VIDEO_LEVEL_BASELINE (1 << 0)
> #define AST_VIDEO_LEVEL_MAIN (1 << 1)
> #define AST_VIDEO_LEVEL_EXTENDED (1 << 2)
>
> #define MAX_VIDEO_SIZES 11
>
> /* The following array defines the video sizes we know about */
> struct videoSize{
> char* type;
> unsigned int num_mbs;
> unsigned int maxframterate;
> };
>
> static struct videoSize videoSizes[MAX_VIDEO_SIZES] = {
> {"SQCIF", 48, 0},
> {"QSIF", 75, 0},
> {"QCIF", 99, 0},
> {"QVGA", 300, 0},
> {"SIF", 300, 0},
> {"CIF", 396, 0},
> {"VGA", 1200, 0},
> {"CIF4", 1584, 0},
> {"SVGA", 1875, 0},
> {"XGA", 2304, 0},
> {"CIF16", 6336, 0},
> };
>
> /*! \brief ast_video_cap: Struct holding video capabilities */ struct
> ast_h2613_video_cap {
> int valid;
> /*!< 1 = This codec is specified, 0 = not specified */
> unsigned int annexes;
> /*!< Video annexes - stored as bitfield 2^(annex-ascii(A))*/
> unsigned int maxframerates[MAX_VIDEO_SIZES]; /*!< Maximum
> framerate (framerate = 30/mpi) */
> unsigned int maxbr;
> /*!< Maximum supported bitrate for this capability */
> };
>
> /*! \brief ast_h264_video_cap: Struct holding video capabilities */
> struct ast_h264_video_cap {
> unsigned int flags; /*!< Video flags
> */
> unsigned int level; /*!< Profile
> level */
> unsigned int maxmbps; /*!< Maximum number of
> macroblocks per second */
> unsigned int minfs; /*!< Minimum
> frame size in macroblocks */
> unsigned int maxfs; /*!< Maximum
> frame size in macroblocks */
> unsigned int maxcpb; /*!< Maximum coded
> picture buffer size in units of 1000 bits (1200 NAL HRD)*/
> unsigned int maxdpb; /*!< Maximum decoded
> picture buffer size in units of 1024 bytes */
> unsigned int packet_mode; /*!< Packetisation mode */
> unsigned int maxbr; /*!< Maximum
> supported bitrate for this capability */
> };
>
> /*! \brief ast_video_capabilities: Struct holding all video capabilities
> */ struct ast_video_capabilities {
> unsigned int maxbitrate; /*!< Max video bitrate */
> struct ast_h2613_video_cap h261; /*!< H261 video capabilities
> */
> struct ast_h2613_video_cap h263; /*!< H263 video capabilities
> */
> struct ast_h264_video_cap h264; /*!< H264 video capabilities
> */ };
>
> /*! \brief ast_video_capabilities: Struct holding all video capabilities
> */ struct ast_capabilities {
> int maxcallbitrate;
> /*!< Max allowed bitrate for this channel */
> struct ast_video_capabilities videocaps; /*!< video
> capabilities */ };
>
> I think that's enough for now... if I've still got any of you with me
> then perhaps the future of what we're up to can come another time....
>
> John
>
>
> John Martin
> John.Martin at AuPix.com
> http://www.AuPix.com
>
> -----Original Message-----
> From: Olle E Johansson [mailto:registry at webway.se]
> Sent: 16 May 2006 10:13
> To: asterisk-video at edvina.net
> Subject: AVTF - What's going on?
>
> This is stuff going on that I'm aware of:
>
> * I am working on fixes for SIP/SDP in the sdpcleanup branch.
> Grandstream and Foniris generously
> donated video phones for me to test with. I am implementing some
> checks if we really have
> video on the incoming call before we offer it on the outbound
> call. Seems stupid to offer
> video without a stream. If we can fix this stuff and test it
> quickly, I can get it into 1.4.
>
> * John Martin contributed some patches for FMTP support that is
> needed. Will open a branch
> for this hopefully this week. This is too late for 1.4, but will
> have to be part of 1.6. It will require
> some architectural changes, and we're beyond freeze for that.
>
> * John Martin is also doing work on H.323 and video
>
> What else is going on codewise or should be added to the wish list?
>
> /Olle
>
>
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
>
> asterisk-video mailing list
> To UNSUBSCRIBE or update options visit:
> http://lists.digium.com/mailman/listinfo/asterisk-video
>
More information about the asterisk-video
mailing list