<p>Joshua Colp has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.asterisk.org/8783">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">bridge_softmix / app_confbridge: Add support for REMB combining.<br><br>This change adds the ability for multiple REMB reports in<br>bridge_softmix to be combined according to a configured<br>behavior into a single report. This single report is sent<br>back to the sender of video, which adjusts the encoding bitrate<br>to be at or below the bitrate of the report. The available<br>behaviors are: lowest, highest, and average. Lowest uses the<br>lowest received bitrate. Highest uses the highest received<br>bitrate. Average goes through the received bitrates adding<br>them to the previous average and creates a new average.<br><br>Other behaviors can be added in the future and the existing<br>average one may be adjusted, but this provides the foundation<br>to do so.<br><br>Support for configuring which behavior to use has been<br>added to app_confbridge.<br><br>ASTERISK-27804<br><br>Change-Id: I9eafe4e7c1f72d67074a8d6acb26bfcf19322b66<br>---<br>M apps/app_confbridge.c<br>M apps/confbridge/conf_config_parser.c<br>M apps/confbridge/include/confbridge.h<br>M bridges/bridge_softmix.c<br>M bridges/bridge_softmix/include/bridge_softmix_internal.h<br>M configs/samples/confbridge.conf.sample<br>M include/asterisk/bridge.h<br>M main/bridge.c<br>8 files changed, 370 insertions(+), 7 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.asterisk.org:29418/asterisk refs/changes/83/8783/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">diff --git a/apps/app_confbridge.c b/apps/app_confbridge.c<br>index d8407d8..25cf275 100644<br>--- a/apps/app_confbridge.c<br>+++ b/apps/app_confbridge.c<br>@@ -1544,6 +1544,13 @@<br>                    ast_bridge_set_sfu_video_mode(conference->bridge);<br>                         ast_bridge_set_video_update_discard(conference->bridge, conference->b_profile.video_update_discard);<br>                    ast_bridge_set_remb_send_interval(conference->bridge, conference->b_profile.remb_send_interval);<br>+                       if (ast_test_flag(&conference->b_profile, BRIDGE_OPT_REMB_BEHAVIOR_AVERAGE)) {<br>+                                ast_brige_set_remb_behavior(conference->bridge, AST_BRIDGE_VIDEO_SFU_REMB_AVERAGE);<br>+                       } else if (ast_test_flag(&conference->b_profile, BRIDGE_OPT_REMB_BEHAVIOR_LOWEST)) {<br>+                          ast_brige_set_remb_behavior(conference->bridge, AST_BRIDGE_VIDEO_SFU_REMB_LOWEST);<br>+                        } else if (ast_test_flag(&conference->b_profile, BRIDGE_OPT_REMB_BEHAVIOR_HIGHEST)) {<br>+                         ast_brige_set_remb_behavior(conference->bridge, AST_BRIDGE_VIDEO_SFU_REMB_HIGHEST);<br>+                       }<br>             }<br> <br>          /* Link it into the conference bridges container */<br>diff --git a/apps/confbridge/conf_config_parser.c b/apps/confbridge/conf_config_parser.c<br>index f9d7483..c143e39 100644<br>--- a/apps/confbridge/conf_config_parser.c<br>+++ b/apps/confbridge/conf_config_parser.c<br>@@ -470,6 +470,27 @@<br>                                            better quality for all receivers.<br>                                     </para></description><br>                             </configOption><br>+                                <configOption name="remb_behavior" default="average"><br>+                                      <synopsis>Sets how REMB reports are generated from multiple sources</synopsis><br>+                                   <description><para><br>+                                              Sets how REMB reports are combined from multiple sources to form one. A REMB report<br>+                                          consists of information about the receiver estimated maximum bitrate. As a source<br>+                                            stream may be forwarded to multiple receivers the reports must be combined into<br>+                                              a single one which is sent to the sender.</para><br>+                                               <enumlist><br>+                                                     <enum name="average"><br>+                                                                <para>The average of all estimated maximum bitrates is taken and sent<br>+                                                          to the sender.</para><br>+                                                  </enum><br>+                                                        <enum name="lowest"><br>+                                                         <para>The lowest estimated maximum bitrate is forwarded to the sender.</para><br>+                                                    </enum><br>+                                                        <enum name="highest"><br>+                                                                <para>The highest estimated maximum bitrate is forwarded to the sender.</para><br>+                                                   </enum><br>+                                                </enumlist><br>+                                    </description><br>+                         </configOption><br>                                 <configOption name="template"><br>                                        <synopsis>When using the CONFBRIDGE dialplan function, use a bridge profile as a template for creating a new temporary profile</synopsis><br>                                 </configOption><br>@@ -1675,6 +1696,23 @@<br>         ast_cli(a->fd,"Video Update Discard: %u\n", b_profile.video_update_discard);<br>     ast_cli(a->fd,"REMB Send Interval: %u\n", b_profile.remb_send_interval);<br> <br>+     switch (b_profile.flags<br>+              & (BRIDGE_OPT_REMB_BEHAVIOR_AVERAGE | BRIDGE_OPT_REMB_BEHAVIOR_LOWEST<br>+                    | BRIDGE_OPT_REMB_BEHAVIOR_HIGHEST)) {<br>+       case BRIDGE_OPT_REMB_BEHAVIOR_AVERAGE:<br>+               ast_cli(a->fd, "REMB Behavior:           average\n");<br>+           break;<br>+       case BRIDGE_OPT_REMB_BEHAVIOR_LOWEST:<br>+                ast_cli(a->fd, "REMB Behavior:           lowest\n");<br>+            break;<br>+       case BRIDGE_OPT_REMB_BEHAVIOR_HIGHEST:<br>+               ast_cli(a->fd, "REMB Behavior:           highest\n");<br>+           break;<br>+       default:<br>+             ast_assert(0);<br>+               break;<br>+       }<br>+<br>  ast_cli(a->fd,"sound_only_person:    %s\n", conf_get_sound(CONF_SOUND_ONLY_PERSON, b_profile.sounds));<br>   ast_cli(a->fd,"sound_only_one:       %s\n", conf_get_sound(CONF_SOUND_ONLY_ONE, b_profile.sounds));<br>      ast_cli(a->fd,"sound_has_joined:     %s\n", conf_get_sound(CONF_SOUND_HAS_JOINED, b_profile.sounds));<br>@@ -2011,6 +2049,30 @@<br>    return 0;<br> }<br> <br>+static int remb_behavior_handler(const struct aco_option *opt, struct ast_variable *var, void *obj)<br>+{<br>+   struct bridge_profile *b_profile = obj;<br>+<br>+   if (strcasecmp(var->name, "remb_behavior")) {<br>+           return -1;<br>+   }<br>+<br>+ ast_clear_flag(b_profile, BRIDGE_OPT_REMB_BEHAVIOR_AVERAGE |<br>+         BRIDGE_OPT_REMB_BEHAVIOR_LOWEST |<br>+            BRIDGE_OPT_REMB_BEHAVIOR_HIGHEST);<br>+<br>+        if (!strcasecmp(var->value, "average")) {<br>+               ast_set_flag(b_profile, BRIDGE_OPT_REMB_BEHAVIOR_AVERAGE);<br>+   } else if (!strcasecmp(var->value, "lowest")) {<br>+         ast_set_flag(b_profile, BRIDGE_OPT_REMB_BEHAVIOR_LOWEST);<br>+    } else if (!strcasecmp(var->value, "highest")) {<br>+                ast_set_flag(b_profile, BRIDGE_OPT_REMB_BEHAVIOR_HIGHEST);<br>+   } else {<br>+             return -1;<br>+   }<br>+    return 0;<br>+}<br>+<br> static int user_template_handler(const struct aco_option *opt, struct ast_variable *var, void *obj)<br> {<br>    struct user_profile *u_profile = obj;<br>@@ -2245,6 +2307,7 @@<br>  aco_option_register_custom(&cfg_info, "sound_", ACO_PREFIX, bridge_types, NULL, sound_option_handler, 0);<br>       aco_option_register(&cfg_info, "video_update_discard", ACO_EXACT, bridge_types, "2000", OPT_UINT_T, 0, FLDSET(struct bridge_profile, video_update_discard));<br>  aco_option_register(&cfg_info, "remb_send_interval", ACO_EXACT, bridge_types, "0", OPT_UINT_T, 0, FLDSET(struct bridge_profile, remb_send_interval));<br>+        aco_option_register_custom(&cfg_info, "remb_behavior", ACO_EXACT, bridge_types, "average", remb_behavior_handler, 0);<br>         /* This option should only be used with the CONFBRIDGE dialplan function */<br>   aco_option_register_custom(&cfg_info, "template", ACO_EXACT, bridge_types, NULL, bridge_template_handler, 0);<br> <br>diff --git a/apps/confbridge/include/confbridge.h b/apps/confbridge/include/confbridge.h<br>index c2f8f9a..0a0a571 100644<br>--- a/apps/confbridge/include/confbridge.h<br>+++ b/apps/confbridge/include/confbridge.h<br>@@ -76,6 +76,9 @@<br>    BRIDGE_OPT_RECORD_FILE_TIMESTAMP = (1 << 5), /*!< Set if the record file should have a timestamp appended */<br>         BRIDGE_OPT_BINAURAL_ACTIVE = (1 << 6), /*!< Set if binaural convolution is activated */<br>      BRIDGE_OPT_VIDEO_SRC_SFU = (1 << 7), /*!< Selective forwarding unit */<br>+      BRIDGE_OPT_REMB_BEHAVIOR_AVERAGE = (1 << 8), /*!< The average of all REMB reports is sent to the sender */<br>+  BRIDGE_OPT_REMB_BEHAVIOR_LOWEST = (1 << 9), /*!< The lowest estimated maximum bitrate is sent to the sender */<br>+      BRIDGE_OPT_REMB_BEHAVIOR_HIGHEST = (1 << 10), /*!< The highest estimated maximum bitrate is sent to the sender */<br> };<br> <br> enum conf_menu_action_id {<br>diff --git a/bridges/bridge_softmix.c b/bridges/bridge_softmix.c<br>index 16e1fb8..ca6527d 100644<br>--- a/bridges/bridge_softmix.c<br>+++ b/bridges/bridge_softmix.c<br>@@ -69,6 +69,15 @@<br> #define SOFTBRIDGE_VIDEO_DEST_LEN strlen(SOFTBRIDGE_VIDEO_DEST_PREFIX)<br> #define SOFTBRIDGE_VIDEO_DEST_SEPARATOR '_'<br> <br>+struct softmix_remb_collector {<br>+     /*! The frame which will be given to each source stream */<br>+   struct ast_frame frame;<br>+      /*! The REMB to send to the source which is collecting REMB reports */<br>+       struct ast_rtp_rtcp_feedback feedback;<br>+       /*! The maximum bitrate */<br>+   unsigned int bitrate;<br>+};<br>+<br> struct softmix_stats {<br>        /*! Each index represents a sample rate used above the internal rate. */<br>      unsigned int sample_rates[16];<br>@@ -768,6 +777,10 @@<br> <br>       ast_stream_topology_free(sc->topology);<br> <br>+        ao2_cleanup(sc->remb_collector);<br>+<br>+       AST_VECTOR_FREE(&sc->video_sources);<br>+<br>        /* Drop mutex lock */<br>         ast_mutex_destroy(&sc->lock);<br> <br>@@ -1140,8 +1153,8 @@<br>                clear_talking(bridge_channel);<br>                break;<br>        case AST_CONTROL_VIDUPDATE:<br>-          if (!bridge->softmix.video_mode.video_update_discard ||<br>-                   ast_tvdiff_ms(ast_tvnow(), softmix_data->last_video_update) > bridge->softmix.video_mode.video_update_discard) {<br>+            if (!bridge->softmix.video_mode.mode_data.sfu_data.video_update_discard ||<br>+                        ast_tvdiff_ms(ast_tvnow(), softmix_data->last_video_update) > bridge->softmix.video_mode.mode_data.sfu_data.video_update_discard) {<br>                  ast_bridge_queue_everyone_else(bridge, NULL, frame);<br>                  softmix_data->last_video_update = ast_tvnow();<br>             }<br>@@ -1156,6 +1169,38 @@<br>     }<br> <br>  return 0;<br>+}<br>+<br>+/*!<br>+ * \internal<br>+ * \brief Determine what to do with an RTCP frame.<br>+ * \since 15.4.0<br>+ *<br>+ * \param bridge Which bridge is getting the frame<br>+ * \param bridge_channel Which channel is writing the frame.<br>+ * \param frame What is being written.<br>+ */<br>+static void softmix_bridge_write_rtcp(struct ast_bridge *bridge, struct ast_bridge_channel *bridge_channel, struct ast_frame *frame)<br>+{<br>+     struct ast_rtp_rtcp_feedback *feedback = frame->data.ptr;<br>+ struct softmix_channel *sc = bridge_channel->tech_pvt;<br>+<br>+ /* We only care about REMB reports right now. In the future we may be able to use sender or<br>+   * receiver reports to further tweak things, but not yet.<br>+     */<br>+  if (frame->subclass.integer != AST_RTP_RTCP_PSFB || feedback->fmt != AST_RTP_RTCP_FMT_REMB ||<br>+          !bridge->softmix.video_mode.mode_data.sfu_data.remb_send_interval) {<br>+              return;<br>+      }<br>+<br>+ /* REMB is the total estimated maximum bitrate across all streams within the session, so we store<br>+     * only the latest report and use it everywhere.<br>+      */<br>+  ast_mutex_lock(&sc->lock);<br>+    sc->remb = feedback->remb;<br>+     ast_mutex_unlock(&sc->lock);<br>+<br>+       return;<br> }<br> <br> /*!<br>@@ -1204,6 +1249,9 @@<br>   case AST_FRAME_CONTROL:<br>               res = softmix_bridge_write_control(bridge, bridge_channel, frame);<br>            break;<br>+       case AST_FRAME_RTCP:<br>+         softmix_bridge_write_rtcp(bridge, bridge_channel, frame);<br>+            break;<br>        case AST_FRAME_BRIDGE_ACTION:<br>                 res = ast_bridge_queue_everyone_else(bridge, bridge_channel, frame);<br>          break;<br>@@ -1217,6 +1265,104 @@<br>       }<br> <br>  return res;<br>+}<br>+<br>+static void remb_collect_report(struct ast_bridge *bridge, struct ast_bridge_channel *bridge_channel,<br>+   struct softmix_bridge_data *softmix_data, struct softmix_channel *sc)<br>+{<br>+    int i;<br>+       unsigned int bitrate;<br>+<br>+     /* If there are no video sources that we are a receiver of then we have noone to<br>+      * report REMB to.<br>+    */<br>+  if (!AST_VECTOR_SIZE(&sc->video_sources)) {<br>+           return;<br>+      }<br>+<br>+ /* We evenly divide the available maximum bitrate across the video sources<br>+    * to this receiver so each source gets an equal slice.<br>+       */<br>+  bitrate = (sc->remb.br_mantissa << sc->remb.br_exp) / AST_VECTOR_SIZE(&sc->video_sources);<br>+<br>+     /* If this receiver has no bitrate yet ignore it */<br>+  if (!bitrate) {<br>+              return;<br>+      }<br>+<br>+ for (i = 0; i < AST_VECTOR_SIZE(&sc->video_sources); ++i) {<br>+                struct softmix_remb_collector *collector;<br>+<br>+         /* The collector will always exist if a video source is in our list */<br>+               collector = AST_VECTOR_GET(&softmix_data->remb_collectors, AST_VECTOR_GET(&sc->video_sources, i));<br>+<br>+          if (!collector->bitrate) {<br>+                        collector->bitrate = bitrate;<br>+                     continue;<br>+            }<br>+<br>+         switch (bridge->softmix.video_mode.mode_data.sfu_data.remb_behavior) {<br>+            case AST_BRIDGE_VIDEO_SFU_REMB_AVERAGE:<br>+                      collector->bitrate = (collector->bitrate + bitrate) / 2;<br>+                       break;<br>+               case AST_BRIDGE_VIDEO_SFU_REMB_LOWEST:<br>+                       if (bitrate < collector->bitrate) {<br>+                            collector->bitrate = bitrate;<br>+                     }<br>+                    break;<br>+               case AST_BRIDGE_VIDEO_SFU_REMB_HIGHEST:<br>+                      if (bitrate > collector->bitrate) {<br>+                            collector->bitrate = bitrate;<br>+                     }<br>+                    break;<br>+               }<br>+    }<br>+}<br>+<br>+static void remb_send_report(struct ast_bridge_channel *bridge_channel, struct softmix_channel *sc)<br>+{<br>+   int i;<br>+<br>+    if (!sc->remb_collector) {<br>+                return;<br>+      }<br>+<br>+ /* If we have a new bitrate then use it for the REMB, if not we use the previous<br>+      * one until we know otherwise. This way the bitrate doesn't drop to 0 all of a sudden.<br>+   */<br>+  if (sc->remb_collector->bitrate) {<br>+             sc->remb_collector->feedback.remb.br_mantissa = sc->remb_collector->bitrate;<br>+             sc->remb_collector->feedback.remb.br_exp = 0;<br>+<br>+               /* The mantissa only has 18 bits available, so while it exceeds them we bump<br>+          * up the exp.<br>+                */<br>+          while (sc->remb_collector->feedback.remb.br_mantissa > 0x3ffff) {<br>+                   sc->remb_collector->feedback.remb.br_mantissa = sc->remb_collector->feedback.remb.br_mantissa >> 1;<br>+                        sc->remb_collector->feedback.remb.br_exp++;<br>+            }<br>+    }<br>+<br>+ for (i = 0; i < AST_VECTOR_SIZE(&bridge_channel->stream_map.to_bridge); ++i) {<br>+             int bridge_num = AST_VECTOR_GET(&bridge_channel->stream_map.to_bridge, i);<br>+<br>+         /* If this stream is not being provided to the bridge there can be no receivers of it<br>+                 * so therefore no REMB reports.<br>+              */<br>+          if (bridge_num == -1) {<br>+                      continue;<br>+            }<br>+<br>+         /* We need to update the frame with this stream, or else it won't be<br>+              * properly routed. We don't use the actual channel stream identifier as<br>+          * the bridging core will do the translation from bridge stream identifier to<br>+                 * channel stream identifier.<br>+                 */<br>+          sc->remb_collector->frame.stream_num = bridge_num;<br>+             ast_bridge_channel_queue_frame(bridge_channel, &sc->remb_collector->frame);<br>+        }<br>+<br>+ sc->remb_collector->bitrate = 0;<br> }<br> <br> static void gather_softmix_stats(struct softmix_stats *stats,<br>@@ -1440,6 +1586,7 @@<br>          struct ast_format *cur_slin = ast_format_cache_get_slin_by_rate(softmix_data->internal_rate);<br>              unsigned int softmix_samples = SOFTMIX_SAMPLES(softmix_data->internal_rate, softmix_data->internal_mixing_interval);<br>            unsigned int softmix_datalen = SOFTMIX_DATALEN(softmix_data->internal_rate, softmix_data->internal_mixing_interval);<br>+           int remb_update = 0;<br> <br>               if (softmix_datalen > MAX_DATALEN) {<br>                       /* This should NEVER happen, but if it does we need to know about it. Almost<br>@@ -1478,6 +1625,13 @@<br>          check_binaural_position_change(bridge, softmix_data);<br> #endif<br> <br>+            /* If we need to do a REMB update to all video sources then do so */<br>+         if (bridge->softmix.video_mode.mode_data.sfu_data.remb_send_interval &&<br>+                   ast_tvdiff_ms(ast_tvnow(), softmix_data->last_remb_update) > bridge->softmix.video_mode.mode_data.sfu_data.remb_send_interval) {<br>+                    remb_update = 1;<br>+                     softmix_data->last_remb_update = ast_tvnow();<br>+             }<br>+<br>          /* Go through pulling audio from each factory that has it available */<br>                AST_LIST_TRAVERSE(&bridge->channels, bridge_channel, entry) {<br>                  struct softmix_channel *sc = bridge_channel->tech_pvt;<br>@@ -1511,6 +1665,9 @@<br>                                              ast_channel_name(bridge_channel->chan));<br> #endif<br>                          mixing_array.used_entries++;<br>+                 }<br>+                    if (remb_update) {<br>+                           remb_collect_report(bridge, bridge_channel, softmix_data, sc);<br>                        }<br>                     ast_mutex_unlock(&sc->lock);<br>           }<br>@@ -1562,6 +1719,10 @@<br> <br>                  /* A frame is now ready for the channel. */<br>                   ast_bridge_channel_queue_frame(bridge_channel, &sc->write_frame);<br>+<br>+                  if (remb_update) {<br>+                           remb_send_report(bridge_channel, sc);<br>+                        }<br>             }<br> <br>          update_all_rates = 0;<br>@@ -1688,6 +1849,8 @@<br>  }<br>     ast_mutex_destroy(&softmix_data->lock);<br>        ast_cond_destroy(&softmix_data->cond);<br>+        AST_VECTOR_RESET(&softmix_data->remb_collectors, ao2_cleanup);<br>+        AST_VECTOR_FREE(&softmix_data->remb_collectors);<br>       ast_free(softmix_data);<br> }<br> <br>@@ -1717,6 +1880,8 @@<br>         softmix_data->default_sample_size = SOFTMIX_SAMPLES(softmix_data->internal_rate,<br>                        softmix_data->internal_mixing_interval);<br> #endif<br>+<br>+      AST_VECTOR_INIT(&softmix_data->remb_collectors, 0);<br> <br>         bridge->tech_pvt = softmix_data;<br> <br>@@ -1814,12 +1979,67 @@<br> <br>                    stream = ast_stream_topology_get_stream(topology, i);<br>                         if (is_video_dest(stream, source_channel_name, source_stream_name)) {<br>+                                struct softmix_channel *sc = participant->tech_pvt;<br>+<br>                             AST_VECTOR_REPLACE(&participant->stream_map.to_channel, bridge_stream_position, i);<br>+                           AST_VECTOR_APPEND(&sc->video_sources, bridge_stream_position);<br>                                 break;<br>                        }<br>             }<br>             ast_channel_unlock(participant->chan);<br>             ast_bridge_channel_unlock(participant);<br>+      }<br>+}<br>+<br>+/*!<br>+ * \brief Allocate a REMB collector<br>+ *<br>+ * \retval non-NULL success<br>+ * \retval NULL failure<br>+ */<br>+static struct softmix_remb_collector *remb_collector_alloc(void)<br>+{<br>+       struct softmix_remb_collector *collector;<br>+<br>+ collector = ao2_alloc_options(sizeof(*collector), NULL, AO2_ALLOC_OPT_LOCK_NOLOCK);<br>+  if (!collector) {<br>+            return NULL;<br>+ }<br>+<br>+ collector->frame.frametype = AST_FRAME_RTCP;<br>+      collector->frame.subclass.integer = AST_RTP_RTCP_PSFB;<br>+    collector->feedback.fmt = AST_RTP_RTCP_FMT_REMB;<br>+  collector->frame.data.ptr = &collector->feedback;<br>+  collector->frame.datalen = sizeof(collector->feedback);<br>+<br>+     return collector;<br>+}<br>+<br>+/*!<br>+ * \brief Setup REMB collection for a particular bridge stream and channel.<br>+ *<br>+ * \param bridge The bridge<br>+ * \param bridge_channel Channel that is collecting REMB information<br>+ * \param bridge_stream_position The slot in the bridge where source video comes from<br>+ */<br>+static void remb_enable_collection(struct ast_bridge *bridge, struct ast_bridge_channel *bridge_channel,<br>+      size_t bridge_stream_position)<br>+{<br>+   struct softmix_channel *sc = bridge_channel->tech_pvt;<br>+    struct softmix_bridge_data *softmix_data = bridge->tech_pvt;<br>+<br>+   if (!sc->remb_collector) {<br>+                sc->remb_collector = remb_collector_alloc();<br>+              if (!sc->remb_collector) {<br>+                        /* This is not fatal. Things will still continue to work but we won't<br>+                     * produce a REMB report to the sender.<br>+                       */<br>+                  return;<br>+              }<br>+    }<br>+<br>+ if (AST_VECTOR_REPLACE(&softmix_data->remb_collectors, bridge_stream_position, ao2_bump(sc->remb_collector))) {<br>+            ao2_ref(sc->remb_collector, -1);<br>   }<br> }<br> <br>@@ -1835,6 +2055,8 @@<br>  */<br> static void softmix_bridge_stream_topology_changed(struct ast_bridge *bridge, struct ast_bridge_channel *bridge_channel)<br> {<br>+ struct softmix_bridge_data *softmix_data = bridge->tech_pvt;<br>+      struct softmix_channel *sc;<br>   struct ast_bridge_channel *participant;<br>       struct ast_vector_int media_types;<br>    int nths[AST_MEDIA_TYPE_END] = {0};<br>@@ -1852,11 +2074,22 @@<br> <br>       AST_VECTOR_INIT(&media_types, AST_MEDIA_TYPE_END);<br> <br>+    /* The bridge stream identifiers may change, so reset the mapping for them.<br>+   * When channels end up getting added back in they'll reuse their existing<br>+        * collector and won't need to allocate a new one (unless they were just added).<br>+  */<br>+  AST_VECTOR_RESET(&softmix_data->remb_collectors, ao2_cleanup);<br>+<br>      /* First traversal: re-initialize all of the participants' stream maps */<br>         AST_LIST_TRAVERSE(&bridge->channels, participant, entry) {<br>             ast_bridge_channel_lock(participant);<br>+<br>              AST_VECTOR_RESET(&participant->stream_map.to_channel, AST_VECTOR_ELEM_CLEANUP_NOOP);<br>           AST_VECTOR_RESET(&participant->stream_map.to_bridge, AST_VECTOR_ELEM_CLEANUP_NOOP);<br>+<br>+                sc = participant->tech_pvt;<br>+               AST_VECTOR_RESET(&sc->video_sources, AST_VECTOR_ELEM_CLEANUP_NOOP);<br>+<br>                 ast_bridge_channel_unlock(participant);<br>       }<br> <br>@@ -1897,7 +2130,12 @@<br>                  if (is_video_source(stream)) {<br>                                AST_VECTOR_APPEND(&media_types, AST_MEDIA_TYPE_VIDEO);<br>                            AST_VECTOR_REPLACE(&participant->stream_map.to_bridge, i, AST_VECTOR_SIZE(&media_types) - 1);<br>-                             AST_VECTOR_REPLACE(&participant->stream_map.to_channel, AST_VECTOR_SIZE(&media_types) - 1, -1);<br>+                           /*<br>+                            * There are cases where we need to bidirectionally send frames, such as for REMB reports<br>+                             * so we also map back to the channel.<br>+                                */<br>+                          AST_VECTOR_REPLACE(&participant->stream_map.to_channel, AST_VECTOR_SIZE(&media_types) - 1, i);<br>+                            remb_enable_collection(bridge, participant, AST_VECTOR_SIZE(&media_types) - 1);<br>                           /*<br>                             * Unlock the channel and participant to prevent<br>                               * potential deadlock in map_source_to_destinations().<br>diff --git a/bridges/bridge_softmix/include/bridge_softmix_internal.h b/bridges/bridge_softmix/include/bridge_softmix_internal.h<br>index f842acb..3aa9091 100644<br>--- a/bridges/bridge_softmix/include/bridge_softmix_internal.h<br>+++ b/bridges/bridge_softmix/include/bridge_softmix_internal.h<br>@@ -50,6 +50,8 @@<br> #include "asterisk/astobj2.h"<br> #include "asterisk/timing.h"<br> #include "asterisk/translate.h"<br>+#include "asterisk/rtp_engine.h"<br>+#include "asterisk/vector.h"<br> <br> #ifdef BINAURAL_RENDERING<br> #include <fftw3.h><br>@@ -124,6 +126,8 @@<br>        int energy_average;<br> };<br> <br>+struct softmix_remb_collector;<br>+<br> /*! \brief Structure which contains per-channel mixing information */<br> struct softmix_channel {<br>    /*! Lock to protect this structure */<br>@@ -169,6 +173,12 @@<br>   struct video_follow_talker_data video_talker;<br>         /*! The ideal stream topology for the channel */<br>      struct ast_stream_topology *topology;<br>+        /*! The latest REMB report from this participant */<br>+  struct ast_rtp_rtcp_feedback_remb remb;<br>+      /*! The REMB collector for this channel, collects REMB from all video receivers */<br>+   struct softmix_remb_collector *remb_collector;<br>+       /*! The bridge streams which are feeding us video sources */<br>+ AST_VECTOR(, int) video_sources;<br> };<br> <br> struct softmix_bridge_data {<br>@@ -202,6 +212,10 @@<br>         unsigned int binaural_init;<br>   /*! The last time a video update was sent into the bridge */<br>  struct timeval last_video_update;<br>+    /*! The last time a REMB frame was sent to each source of video */<br>+   struct timeval last_remb_update;<br>+     /*! Per-bridge stream REMB collectors, which flow back to video source */<br>+    AST_VECTOR(, struct softmix_remb_collector *) remb_collectors;<br> };<br> <br> struct softmix_mixing_array {<br>diff --git a/configs/samples/confbridge.conf.sample b/configs/samples/confbridge.conf.sample<br>index 4028593..8b276cd 100644<br>--- a/configs/samples/confbridge.conf.sample<br>+++ b/configs/samples/confbridge.conf.sample<br>@@ -239,6 +239,10 @@<br>                            ; A REMB frame contains receiver estimated maximum bitrate information. By creating a combined<br>                            ; frame and sending it to the sources of video the sender can be influenced on what bitrate<br>                            ; they choose allowing a better experience for the receivers. This defaults to 0, or disabled.<br>+;remb_behavior=average     ; How the combined REMB report for an SFU video bridge is constructed. If set to "average" then<br>+                           ; the estimated maximum bitrate of each receiver is used to construct an average bitrate. If<br>+                           ; set to "lowest" the lowest maximum bitrate is forwarded to the sender. If set to "highest"<br>+                           ; the highest maximum bitrate is forwarded to the sender. This defaults to "average".<br> <br> ; All sounds in the conference are customizable using the bridge profile options below.<br> ; Simply state the option followed by the filename or full path of the filename after<br>diff --git a/include/asterisk/bridge.h b/include/asterisk/bridge.h<br>index c96cefb..2f42f1c 100644<br>--- a/include/asterisk/bridge.h<br>+++ b/include/asterisk/bridge.h<br>@@ -126,6 +126,26 @@<br>    struct ast_channel *chan_old_vsrc;<br> };<br> <br>+/*! \brief REMB report behaviors */<br>+enum ast_bridge_video_sfu_remb_behavior {<br>+ /*! The average of all reports is sent to the sender */<br>+      AST_BRIDGE_VIDEO_SFU_REMB_AVERAGE = 0,<br>+       /*! The lowest reported bitrate is forwarded to the sender */<br>+        AST_BRIDGE_VIDEO_SFU_REMB_LOWEST,<br>+    /*! The highest reported bitrate is forwarded to the sender */<br>+       AST_BRIDGE_VIDEO_SFU_REMB_HIGHEST,<br>+};<br>+<br>+/*! \brief This is used for selective forwarding unit configuration */<br>+struct ast_bridge_video_sfu_data {<br>+     /*! The minimum interval between video updates */<br>+    unsigned int video_update_discard;<br>+   /*! The interval at which a REMB report is generated and sent */<br>+     unsigned int remb_send_interval;<br>+     /*! How the combined REMB report is generated */<br>+     enum ast_bridge_video_sfu_remb_behavior remb_behavior;<br>+};<br>+<br> /*! \brief Data structure that defines a video source mode */<br> struct ast_bridge_video_mode {<br>       enum ast_bridge_video_mode_type mode;<br>@@ -133,9 +153,8 @@<br>    union {<br>               struct ast_bridge_video_single_src_data single_src_data;<br>              struct ast_bridge_video_talker_src_data talker_src_data;<br>+             struct ast_bridge_video_sfu_data sfu_data;<br>    } mode_data;<br>- unsigned int video_update_discard;<br>-   unsigned int remb_send_interval;<br> };<br> <br> /*!<br>@@ -921,6 +940,14 @@<br> void ast_bridge_set_remb_send_interval(struct ast_bridge *bridge, unsigned int remb_send_interval);<br> <br> /*!<br>+ * \brief Set the REMB report generation behavior on a bridge<br>+ *<br>+ * \param bridge Bridge to set the REMB behavior on<br>+ * \param behavior How REMB reports are generated<br>+ */<br>+void ast_brige_set_remb_behavior(struct ast_bridge *bridge, enum ast_bridge_video_sfu_remb_behavior behavior);<br>+<br>+/*!<br>  * \brief Update information about talker energy for talker src video mode.<br>  */<br> void ast_bridge_update_talker_src_video_mode(struct ast_bridge *bridge, struct ast_channel *chan, int talker_energy, int is_keyfame);<br>diff --git a/main/bridge.c b/main/bridge.c<br>index 8795081..14f0181 100644<br>--- a/main/bridge.c<br>+++ b/main/bridge.c<br>@@ -3846,14 +3846,21 @@<br> void ast_bridge_set_video_update_discard(struct ast_bridge *bridge, unsigned int video_update_discard)<br> {<br>     ast_bridge_lock(bridge);<br>-     bridge->softmix.video_mode.video_update_discard = video_update_discard;<br>+   bridge->softmix.video_mode.mode_data.sfu_data.video_update_discard = video_update_discard;<br>         ast_bridge_unlock(bridge);<br> }<br> <br> void ast_bridge_set_remb_send_interval(struct ast_bridge *bridge, unsigned int remb_send_interval)<br> {<br>    ast_bridge_lock(bridge);<br>-     bridge->softmix.video_mode.remb_send_interval = remb_send_interval;<br>+       bridge->softmix.video_mode.mode_data.sfu_data.remb_send_interval = remb_send_interval;<br>+    ast_bridge_unlock(bridge);<br>+}<br>+<br>+void ast_brige_set_remb_behavior(struct ast_bridge *bridge, enum ast_bridge_video_sfu_remb_behavior behavior)<br>+{<br>+        ast_bridge_lock(bridge);<br>+     bridge->softmix.video_mode.mode_data.sfu_data.remb_behavior = behavior;<br>    ast_bridge_unlock(bridge);<br> }<br> <br></pre><p>To view, visit <a href="https://gerrit.asterisk.org/8783">change 8783</a>. To unsubscribe, visit <a href="https://gerrit.asterisk.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.asterisk.org/8783"/><meta itemprop="name" content="View Change"/></div></div>

<div style="display:none"> Gerrit-Project: asterisk </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: I9eafe4e7c1f72d67074a8d6acb26bfcf19322b66 </div>
<div style="display:none"> Gerrit-Change-Number: 8783 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Joshua Colp <jcolp@digium.com> </div>