<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/en/2160/3/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Media+Operations">Media Operations</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~mmichelson">Mark Michelson</a>
</h4>
<br/>
<h4>Changes (1)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" >Let's consider a case dealing with the bridge instead. In this scenario, two session wish to communicate. One session's source audio is G.722 and the sink provided by the other session is G.711 ulaw. The bridge, upon seeing that the two formats are incompatible, will enlist the help of the transcoder. This time, the transcoder gets to be a bit more of a shining hero to the bridge because the transcoder is capable of chaining together several transforming media operations in order to make the translation path necessary. Let's have a look: <br> <br></td></tr>
<tr><td class="diff-changed-lines" ><span class="diff-changed-words">{gliffy:name=bridge_media_flow|align=left|size=L|version=<span class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">2</span><span class="diff-added-chars"style="background-color: #dfd;">3</span>}</span> <br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<div class='panelMacro'><table class='warningMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/wiki/images/icons/emoticons/forbidden.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td>This is a work in progress.
<p>In fact, definitely don't look at this yet. It's kind of a mess.</p></td></tr></table></div>
<h1><a name="MediaOperations-Introduction"></a>Introduction</h1>
<p>In Asterisk SCF, a "media operation" refers to a process by which media is manipulated. Common media operations would be transcoding, jitter buffering, volume adjustment, pitch shifting, and potentially media injection. What follows is a discussion of how media operations fit into the architecture of Asterisk SCF, and how they can be used.</p>
<h1><a name="MediaOperations-AsteriskSCFwithoutmediaoperations"></a>Asterisk SCF without media operations</h1>
<p>Without media operations, Asterisk SCF's media handling is straightforward, but not very effective. Each session is equipped with media stream sources and sinks. A stream source is where audio comes from, and a stream sink is where audio is sent. The bridging component can simply connect a source from one session to a sink of another session and everything works. This can quickly become a problem if two sessions are incapable of speaking the same media format to each other.</p>
<h1><a name="MediaOperations-Classificationofmediaoperations"></a>Classification of media operations</h1>
<p>Media operations can be categorized as follows:</p>
<ul>
<li>Transforming</li>
<li>Adjusting</li>
</ul>
<p>Transforming media operations are those whose purpose is to change the format of media arriving into something else. This may mean changing to a new media format, or it may mean changing the parameters of the current format. Examples of transforming media operations would be transcoding and resampling.</p>
<p>Adjusting media operations are those that make a change to the media without changing its compatibility in any way. Examples of adjusting media operations are jitter buffering and volume adjusting.</p>
<h1><a name="MediaOperations-Useofmediaoperations"></a>Use of media operations</h1>
<p>Media operations are valid to use anywhere that media exists. In practice, this means that there are two levels at which media operations may be used:</p>
<ul>
<li>Session level. Session level media operations will affect media to and/or from a specific session. This can be useful for operations that should not affect all parties in a bridged call. If a conference participant has hearing issues, then it may be good to place a volume adjustment media operation for media going to his session.</li>
</ul>
<ul>
<li>Bridge level. Bridge level media operations will affect media to and/or from all sessions involved in a particular bridge. This type will be more rare.</li>
</ul>
<p>How does a session or bridge know to use media operations? There are two ways to use them:</p>
<ul>
<li>Configuration. Endpoints and bridges may be configured to have specific media operations placed on their sessions or bridges by default.</li>
<li>Hooks. Session creation hooks and bridge creation hooks are ways to dynamically insert media operations for a session or bridge.</li>
</ul>
<h1><a name="MediaOperations-Mediaoperationslice"></a>Media operation slice</h1>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">interface MediaOperation
{
/**
* Get stream source(s) for this operation.
*/
AsteriskSCF::Media::V1::StreamSourceSeq getSources();
/**
* Get stream sink(s) for this operation
*/
AsteriskSCF::Media::V1::StreamSinkSeq getSinks();
};</pre>
</div></div>
<p>The MediaOperation is straightforward. It simply provides sources and sinks for media. Someone simply can write to a media operation sink and have new media come out one of the sources.</p>
<p>So how does one actually acquire a media operation? Via the service locator of course! Just like with most items looked up in the service locator, a media operation is found using a category and optional name. Potential media operation categories would be "pitchShift" or "jitterBuffer".</p>
<h1><a name="MediaOperations-Dealingwithformats"></a>Dealing with formats</h1>
<p>Most media operations will not be able to support any arbitrary media format. It is almost certain that there will be a need to transcode between formats when using a media operation. In order to be able to translate between formats, transcoding support is necessary. Transcoding is a bit tricky since it may take multiple translation steps to get from one format to another. Since each transcoding step would be a single media operation, it becomes difficult to try to request a single transcoding operation. Instead, what is more handy is to have a transcoding service available. This service will maintain a graph of available formats and their ideal translation paths. Given any two formats, the transcoding service can then allocate media operations to satisfy the necessary translation.</p>
<p>The transcoding service will have an interface like the following:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">/**
* Indicates the transcoding service was given an input source or sink with an unknown format
*/
exception UnknownFormat
{
};
/**
* Indicates the trnascoding service is unable to translate between the formats given
*/
exception NoPathAvailable
{
};
interface TranscodingService
{
/**
* Sets up a translation path between a stream source and a stream sink.
*
* @param inSource The initial source of media
* @param inSink The initial destination of media
* @param outSource The new source of media for inSink
* @param outSink The new sink for media from inSource
*/
void getTranslationPath(
AsteriskSCF::Media::V1::StreamSource inSource,
AsteriskSCF::Media::V1::StreamSink inSink,
out AsteriskSCF::Media::V1::StreamSource inSource,
out AsteriskSCF::Media::V1::StreamSink inSink) throws UnknownFormat, throws NoPathAvailable;
/**
* Adds a new transcoder to the service.
* The formats supported by the transcoder can be discerned
* through method calls on the transcoder
* parameter.
*/
void addTranscoder(MediaOperation* transcoder);
/**
* Removes a transcoder from the service.
*/
void removeTranscoder(MediaOperation *transcoder);
};</pre>
</div></div>
<p>With this, any transforming media operation will need to register itself with the transcoding service. This way, when transcoding is necessary, a component can ask the transcoder to set things up for them.</p>
<h1><a name="MediaOperations-Howaboutanexample%3F"></a>How about an example?</h1>
<p>Let's examine a session level example. Let's say that a session gateway creates a session with an endpoint. We will be receiving G.711 ulaw audio from this endpoint. This endpoint has been configured to have a jitter buffer media operation on its incoming media path. The jitter buffer requires 8 kHz signed linear audio as input and outputs the same (since it is an adjusting media operation).</p>
<p>The session gateway, upon setting up the G.711 ulaw stream source for the session, will inspect the configured media operations to employ. First, the jitter buffer is looked up. The jitter buffer is found using the service locator, and its MediaOperation proxy is returned. </p>
<table width="100%">
<tr>
<td align="left">
<table>
<caption align="bottom">
</caption>
<tr>
<td>
<img style="border: none; width: 1211px; height: 397px;"
usemap="#GLIFFY_MAP_17203407_media_flow_mismatch"
src="/wiki/download/attachments/17203407/media_flow_mismatch.png?version=1&modificationDate=1311353214868"
alt="A&#32;Gliffy&#32;Diagram&#32;named&#58;&#32;media&#95;flow&#95;mismatch"/>
</td>
</tr>
</table>
</td>
</tr>
</table>
<p>There is a problem. The G.711 source cannot send audio directly to the 8 kHz signed linear sink of the jitter buffer. The session gateway calls on the trnascoder service to provide a path between G.711 ulaw and 8 kHz signed linear. The transcoder provides an appropriate source and a sink for translating the audio. The session gateway connects the sources and sinks as appropriate.</p>
<table width="100%">
<tr>
<td align="left">
<table>
<caption align="bottom">
</caption>
<tr>
<td>
<img style="border: none; width: 1211px; height: 399px;"
usemap="#GLIFFY_MAP_17203407_media_flow"
src="/wiki/download/attachments/17203407/media_flow.png?version=1&modificationDate=1311352451951"
alt="A&#32;Gliffy&#32;Diagram&#32;named&#58;&#32;media&#95;flow"/>
</td>
</tr>
</table>
</td>
</tr>
</table>
<h1><a name="MediaOperations-Howaboutadifferentexample%3F"></a>How about a different example?</h1>
<p>Let's consider a case dealing with the bridge instead. In this scenario, two session wish to communicate. One session's source audio is G.722 and the sink provided by the other session is G.711 ulaw. The bridge, upon seeing that the two formats are incompatible, will enlist the help of the transcoder. This time, the transcoder gets to be a bit more of a shining hero to the bridge because the transcoder is capable of chaining together several transforming media operations in order to make the translation path necessary. Let's have a look:</p>
<table width="100%">
<tr>
<td align="left">
<table>
<caption align="bottom">
</caption>
<tr>
<td>
<img style="border: none; width: 1334px; height: 549px;"
usemap="#GLIFFY_MAP_17203407_bridge_media_flow"
src="/wiki/download/attachments/17203407/bridge_media_flow.png?version=3&modificationDate=1311354909620"
alt="A&#32;Gliffy&#32;Diagram&#32;named&#58;&#32;bridge&#95;media&#95;flow"/>
</td>
</tr>
</table>
</td>
</tr>
</table>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;" class="grey">
<a href="https://wiki.asterisk.org/wiki/users/removespacenotification.action?spaceKey=TOP">Stop watching space</a>
<span style="padding: 0px 5px;">|</span>
<a href="https://wiki.asterisk.org/wiki/users/editmyemailsettings.action">Change email notification preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Media+Operations">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=17203407&revisedVersion=11&originalVersion=10">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Media+Operations?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>