<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/en/2160/3/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Media+Operations">Media Operations</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~mmichelson">Mark Michelson</a>
</h4>
<br/>
<h4>Changes (1)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-added-lines" style="background-color: #dfd;">{gliffy:name=media_flow_mismatch|align=left|size=L|version=1} <br></td></tr>
<tr><td class="diff-unchanged" > <br>{Warning} <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<table width="100%">
<tr>
<td align="left">
<table>
<caption align="bottom">
</caption>
<tr>
<td>
<img style="border: none; width: 1211px; height: 397px;"
usemap="#GLIFFY_MAP_17203407_media_flow_mismatch"
src="/wiki/download/attachments/17203407/media_flow_mismatch.png?version=1&modificationDate=1311353214868"
alt="A&#32;Gliffy&#32;Diagram&#32;named&#58;&#32;media&#95;flow&#95;mismatch"/>
</td>
</tr>
</table>
</td>
</tr>
</table>
<div class='panelMacro'><table class='warningMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/wiki/images/icons/emoticons/forbidden.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td>This is a work in progress.
<p>In fact, definitely don't look at this yet. It's kind of a mess.</p></td></tr></table></div>
<h1><a name="MediaOperations-Introduction"></a>Introduction</h1>
<p>In Asterisk SCF, a "media operation" refers to a process by which media is manipulated. Common media operations would be transcoding, jitter buffering, volume adjustment, pitch shifting, and potentially media injection. What follows is a discussion of how media operations fit into the architecture of Asterisk SCF, and how they can be used.</p>
<h1><a name="MediaOperations-AsteriskSCFwithoutmediaoperations"></a>Asterisk SCF without media operations</h1>
<p>Without media operations, Asterisk SCF's media handling is straightforward, but not very effective. Each session is equipped with media stream sources and sinks. A stream source is where audio comes from, and a stream sink is where audio is sent. The bridging component can simply connect a source from one session to a sink of another session and everything works. This can quickly become a problem if two sessions are incapable of speaking the same media format to each other.</p>
<h1><a name="MediaOperations-Classificationofmediaoperations"></a>Classification of media operations</h1>
<p>Media operations can be categorized as follows:</p>
<ul>
<li>Transforming</li>
<li>Adjusting</li>
</ul>
<p>Transforming media operations are those whose purpose is to change the format of media arriving into something else. This may mean changing to a new media format, or it may mean changing the parameters of the current format. Examples of transforming media operations would be transcoding and resampling.</p>
<p>Adjusting media operations are those that make a change to the media without changing its compatibility in any way. Examples of adjusting media operations are jitter buffering and volume adjusting.</p>
<h1><a name="MediaOperations-Useofmediaoperations"></a>Use of media operations</h1>
<p>Media operations are valid to use anywhere that media exists. In practice, this means that there are two levels at which media operations may be used:</p>
<ul>
<li>Session level. Session level media operations will affect media to and/or from a specific session. This can be useful for operations that should not affect all parties in a bridged call. If a conference participant has hearing issues, then it may be good to place a volume adjustment media operation for media going to his session.</li>
</ul>
<ul>
<li>Bridge level. Bridge level media operations will affect media to and/or from all sessions involved in a particular bridge. This type will be more rare.</li>
</ul>
<p>How does a session or bridge know to use media operations? There are two ways to use them:</p>
<ul>
<li>Configuration. Endpoints and bridges may be configured to have specific media operations placed on their sessions or bridges by default.</li>
<li>Hooks. Session creation hooks and bridge creation hooks are ways to dynamically insert media operations for a session or bridge.</li>
</ul>
<h1><a name="MediaOperations-Mediaoperationslice"></a>Media operation slice</h1>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">interface MediaOperation
{
/**
* Get stream source(s) for this operation.
*/
AsteriskSCF::Media::V1::StreamSourceSeq getSources();
/**
* Get stream sink(s) for this operation
*/
AsteriskSCF::Media::V1::StreamSinkSeq getSinks();
};</pre>
</div></div>
<p>The MediaOperation is straightforward. It simply provides sources and sinks for media. Someone simply can write to a media operation sink and have new media come out one of the sources.</p>
<p>So how does one actually acquire a media operation? Via the service locator of course! Just like with most items looked up in the service locator, a media operation is found using a category and optional name. Potential media operation categories would be "pitchShift" or "jitterBuffer".</p>
<h1><a name="MediaOperations-Dealingwithformats"></a>Dealing with formats</h1>
<p>Most media operations will not be able to support any arbitrary media format. It is almost certain that there will be a need to transcode between formats when using a media operation. In order to be able to translate between formats, transcoding support is necessary. Transcoding is a bit tricky since it may take multiple translation steps to get from one format to another. Since each transcoding step would be a single media operation, it becomes difficult to try to request a single transcoding operation. Instead, what is more handy is to have a transcoding service available. This service will maintain a graph of available formats and their ideal translation paths. Given any two formats, the transcoding service can then allocate media operations to satisfy the necessary translation.</p>
<p>The transcoding service will have an interface like the following:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="theme: Confluence; brush: java; gutter: false">/**
* Indicates the transcoding service was given an input source or sink with an unknown format
*/
exception UnknownFormat
{
};
/**
* Indicates the trnascoding service is unable to translate between the formats given
*/
exception NoPathAvailable
{
};
interface TranscodingService
{
/**
* Sets up a translation path between a stream source and a stream sink.
*
* @param inSource The initial source of media
* @param inSink The initial destination of media
* @param outSource The new source of media for inSink
* @param outSink The new sink for media from inSource
*/
void getTranslationPath(
AsteriskSCF::Media::V1::StreamSource inSource,
AsteriskSCF::Media::V1::StreamSink inSink,
out AsteriskSCF::Media::V1::StreamSource inSource,
out AsteriskSCF::Media::V1::StreamSink inSink) throws UnknownFormat, throws NoPathAvailable;
/**
* Adds a new transcoder to the service.
* The formats supported by the transcoder can be discerned
* through method calls on the transcoder
* parameter.
*/
void addTranscoder(MediaOperation* transcoder);
/**
* Removes a transcoder from the service.
*/
void removeTranscoder(MediaOperation *transcoder);
};</pre>
</div></div>
<p>With this, any transforming media operation will need to register itself with the transcoding service. This way, when transcoding is necessary, a component can ask the transcoder to set things up for them.</p>
<h1><a name="MediaOperations-Howaboutanexample%3F"></a>How about an example?</h1>
<p>Let's examine a session level example. Let's say that a session gateway creates a session with an endpoint. We will be receiving G.711 ulaw audio from this endpoint. This endpoint has been configured to have two media operations on its incoming media path: jitter buffering and pitch shifting. The jitter buffer requires 8 kHz signed linear audio as input and outputs the same (since it is an adjusting media operation). The pitch shifter can accept either 8 kHz, 16 kHz, or 32 kHz signed linear audio as input and outputs the same type that it received.</p>
<p>The session gateway, upon setting up the G.711 ulaw stream source for the session, will inspect the configured media operations to employ. First, the jitter buffer is looked up. The jitter buffer is found using the service locator, and its MediaOperation proxy is returned. The session gateway checks the G.711 ulaw source against the sink of the jitter buffer media operation and determines that the two are incompatible. The session gateway calls on the trnascoder service to provide a path between G.711 ulaw and 8 kHz signed linear. The transcoder provides an appropriate source and a sink<br/>
for translating the audio.</p>
<table width="100%">
<tr>
<td align="left">
<table>
<caption align="bottom">
</caption>
<tr>
<td>
<img style="border: none; width: 1211px; height: 399px;"
usemap="#GLIFFY_MAP_17203407_media_flow"
src="/wiki/download/attachments/17203407/media_flow.png?version=1&modificationDate=1311352451951"
alt="A&#32;Gliffy&#32;Diagram&#32;named&#58;&#32;media&#95;flow"/>
</td>
</tr>
</table>
</td>
</tr>
</table>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;" class="grey">
<a href="https://wiki.asterisk.org/wiki/users/removespacenotification.action?spaceKey=TOP">Stop watching space</a>
<span style="padding: 0px 5px;">|</span>
<a href="https://wiki.asterisk.org/wiki/users/editmyemailsettings.action">Change email notification preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Media+Operations">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=17203407&revisedVersion=6&originalVersion=5">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Media+Operations?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>