<p>Friendly Automation <strong>submitted</strong> this change.</p><p><a href="https://gerrit.asterisk.org/c/asterisk/+/14603">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Joshua Colp: Looks good to me, but someone else must approve
Kevin Harwell: Looks good to me, approved
Friendly Automation: Approved for Submit
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">res_corosync: Fix crash in huge distributed environment.<br><br>1) Fix memory-leaks<br> Added code to release ast_events extracted from corosync and stasis messages<br><br>2) Clean stasis cache when a member of the corosync cluster leaves the group<br> Added code to remove from the stasis cache of the members remained on the<br> group all the messages with the EID of the left member.<br> If the device states of the left member remain in the stasis cache of other<br> members, they will not be updated anymore and high priority cached values,<br> like BUSY, will take precedence over current device states.<br><br>3) Stop corosync event propagation when node is not joined to the group<br> Updated dispatch_thread_handler code to detect when asterisk is not joined<br> to the corosync group and added some condition in publish_event_to_corosync<br> code to send corosync messages only when joined.<br> When a node is not joined its corosync daemon can't send messages:<br> the cpg_mcast_joined function append new messages to the FIFO buffer until<br> it's full and then it blocks indefinitely.<br> In this scenario if the stasis_message_cb callback, registered by<br> res_corosync to handle stasis messages, try to send a corosync messages,<br> the thread of the stasis thread-pool will be blocked until the node join<br> the corosync cluster.<br><br>ASTERISK-28888<br>Reported by: Università di Bologna - CESIA VoIP<br><br>Change-Id: Ie8e99bc23f141a73c13ae6fb1948d148d4de17f2<br>---<br>M res/res_corosync.c<br>1 file changed, 403 insertions(+), 167 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/res/res_corosync.c b/res/res_corosync.c</span><br><span>index 03f2c5e..e5448b9 100644</span><br><span>--- a/res/res_corosync.c</span><br><span>+++ b/res/res_corosync.c</span><br><span>@@ -46,13 +46,21 @@</span><br><span> #include "asterisk/stasis.h"</span><br><span> #include "asterisk/stasis_message_router.h"</span><br><span> #include "asterisk/stasis_system.h"</span><br><span style="color: hsl(120, 100%, 40%);">+#include "asterisk/taskprocessor.h"</span><br><span> </span><br><span> AST_RWLOCK_DEFINE_STATIC(event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+AST_RWLOCK_DEFINE_STATIC(init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+/*! \brief Timeout for Corosync's poll process */</span><br><span style="color: hsl(120, 100%, 40%);">+#define COROSYNC_POLL_TIMEOUT (10 * 1000)</span><br><span> </span><br><span> static void publish_mwi_to_stasis(struct ast_event *event);</span><br><span> static void publish_device_state_to_stasis(struct ast_event *event);</span><br><span> static void publish_cluster_discovery_to_stasis(struct ast_event *event);</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+/*! \brief Join to corosync */</span><br><span style="color: hsl(120, 100%, 40%);">+static int corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> /*! \brief All the nodes that we're aware of */</span><br><span> static struct ao2_container *nodes;</span><br><span> </span><br><span>@@ -197,6 +205,7 @@</span><br><span> {</span><br><span> struct corosync_ping_payload *payload;</span><br><span> struct stasis_message *message;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct ast_eid *event_eid;</span><br><span> </span><br><span> ast_assert(ast_event_get_type(event) == AST_EVENT_PING);</span><br><span> ast_assert(event != NULL);</span><br><span>@@ -209,7 +218,10 @@</span><br><span> if (!payload) {</span><br><span> return;</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">- payload->event = event;</span><br><span style="color: hsl(120, 100%, 40%);">+ event_eid = (struct ast_eid *)ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span style="color: hsl(120, 100%, 40%);">+ payload->event = ast_event_new(AST_EVENT_PING,</span><br><span style="color: hsl(120, 100%, 40%);">+ AST_EVENT_IE_EID, AST_EVENT_IE_PLTYPE_RAW, event_eid, sizeof(*event_eid),</span><br><span style="color: hsl(120, 100%, 40%);">+ AST_EVENT_IE_END);</span><br><span> </span><br><span> message = stasis_message_create(corosync_ping_message_type(), payload);</span><br><span> if (!message) {</span><br><span>@@ -345,7 +357,7 @@</span><br><span> ast_assert(ast_event_get_type(event) == AST_EVENT_CLUSTER_DISCOVERY);</span><br><span> </span><br><span> event_eid = (struct ast_eid *)ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span style="color: hsl(0, 100%, 40%);">- if (!ast_eid_cmp(&ast_eid_default, event_eid)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!event_eid || !ast_eid_cmp(&ast_eid_default, event_eid)) {</span><br><span> /* Don't feed events back in that originated locally. */</span><br><span> return;</span><br><span> }</span><br><span>@@ -475,6 +487,7 @@</span><br><span> struct ast_event *event;</span><br><span> void (*publish_handler)(struct ast_event *) = NULL;</span><br><span> enum ast_event_type event_type;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct ast_eid *event_eid;</span><br><span> </span><br><span> if (msg_len < ast_event_minimum_length()) {</span><br><span> ast_debug(1, "Ignoring event that's too small. %u < %u\n",</span><br><span>@@ -483,7 +496,8 @@</span><br><span> return;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (!ast_eid_cmp(&ast_eid_default, ast_event_get_ie_raw(msg, AST_EVENT_IE_EID))) {</span><br><span style="color: hsl(120, 100%, 40%);">+ event_eid = (struct ast_eid *)ast_event_get_ie_raw(msg, AST_EVENT_IE_EID);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!event_eid || !ast_eid_cmp(&ast_eid_default, event_eid)) {</span><br><span> /* Don't feed events back in that originated locally. */</span><br><span> return;</span><br><span> }</span><br><span>@@ -495,14 +509,17 @@</span><br><span> }</span><br><span> </span><br><span> ast_rwlock_rdlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_deliver_cb rdlock\n");</span><br><span> publish_handler = event_types[event_type].publish_to_stasis;</span><br><span> if (!event_types[event_type].subscribe || !publish_handler) {</span><br><span> /* We are not configured to subscribe to these events or</span><br><span> we have no way to publish it internally. */</span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_deliver_cb unlock\n");</span><br><span> return;</span><br><span> }</span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_deliver_cb unlock\n");</span><br><span> </span><br><span> if (!(event = ast_malloc(msg_len))) {</span><br><span> return;</span><br><span>@@ -514,13 +531,14 @@</span><br><span> const struct ast_eid *eid;</span><br><span> char buf[128] = "";</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- eid = ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span style="color: hsl(120, 100%, 40%);">+ eid = (struct ast_eid *)ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span> ast_eid_to_str(buf, sizeof(buf), (struct ast_eid *) eid);</span><br><span> ast_log(LOG_NOTICE, "Got event PING from server with EID: '%s'\n", buf);</span><br><span> }</span><br><span> ast_debug(5, "Publishing event %s (%u) to stasis\n",</span><br><span> ast_event_get_type_name(event), event_type);</span><br><span> publish_handler(event);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_free(event);</span><br><span> }</span><br><span> </span><br><span> static void publish_event_to_corosync(struct ast_event *event)</span><br><span>@@ -536,22 +554,32 @@</span><br><span> </span><br><span> /* The stasis subscription will only exist if we are configured to publish</span><br><span> * these events, so just send away. */</span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_mcast_joined(cpg_handle, CPG_TYPE_FIFO, &iov, 1)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_WARNING, "CPG mcast failed (%u) for event %s (%u)\n",</span><br><span style="color: hsl(0, 100%, 40%);">- cs_err, ast_event_get_type_name(event), ast_event_get_type(event));</span><br><span style="color: hsl(120, 100%, 40%);">+ if (corosync_node_joined && !ast_rwlock_tryrdlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "publish_event_to_corosync rdlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_mcast_joined(cpg_handle, CPG_TYPE_FIFO, &iov, 1)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "CPG mcast failed (%u) for event %s (%u)\n",</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err, ast_event_get_type_name(event), ast_event_get_type(event));</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "publish_event_to_corosync unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "CPG mcast not executed for event %s (%u): initializing CPG.\n",</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_event_get_type_name(event), ast_event_get_type(event));</span><br><span> }</span><br><span> }</span><br><span> </span><br><span> static void publish_to_corosync(struct stasis_message *message)</span><br><span> {</span><br><span> struct ast_event *event;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct ast_eid *event_eid;</span><br><span> </span><br><span> event = stasis_message_to_event(message);</span><br><span> if (!event) {</span><br><span> return;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (ast_eid_cmp(&ast_eid_default, ast_event_get_ie_raw(event, AST_EVENT_IE_EID))) {</span><br><span style="color: hsl(120, 100%, 40%);">+ event_eid = (struct ast_eid *)ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!event_eid || ast_eid_cmp(&ast_eid_default, event_eid)) {</span><br><span> /* If the event didn't originate from this server, don't send it back out. */</span><br><span> ast_event_destroy(event);</span><br><span> return;</span><br><span>@@ -561,12 +589,13 @@</span><br><span> const struct ast_eid *eid;</span><br><span> char buf[128] = "";</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- eid = ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span style="color: hsl(120, 100%, 40%);">+ eid = (struct ast_eid *)ast_event_get_ie_raw(event, AST_EVENT_IE_EID);</span><br><span> ast_eid_to_str(buf, sizeof(buf), (struct ast_eid *) eid);</span><br><span> ast_log(LOG_NOTICE, "Sending event PING from this server with EID: '%s'\n", buf);</span><br><span> }</span><br><span> </span><br><span> publish_event_to_corosync(event);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_event_destroy(event);</span><br><span> }</span><br><span> </span><br><span> static void stasis_message_cb(void *data, struct stasis_subscription *sub, struct stasis_message *message)</span><br><span>@@ -591,6 +620,30 @@</span><br><span> return 0;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+static int clear_node_cache(void *obj, void *arg, int flags)</span><br><span style="color: hsl(120, 100%, 40%);">+{</span><br><span style="color: hsl(120, 100%, 40%);">+ struct stasis_message *cached_msg = obj;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct stasis_topic *topic = arg;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct stasis_message *msg;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct ast_eid *msg_eid;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!cached_msg) {</span><br><span style="color: hsl(120, 100%, 40%);">+ return 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ msg_eid = (struct ast_eid *)stasis_message_eid(cached_msg);</span><br><span style="color: hsl(120, 100%, 40%);">+ if(msg_eid && ast_eid_cmp(&ast_eid_default, msg_eid))</span><br><span style="color: hsl(120, 100%, 40%);">+ {</span><br><span style="color: hsl(120, 100%, 40%);">+ msg = stasis_cache_clear_create(cached_msg);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (msg) {</span><br><span style="color: hsl(120, 100%, 40%);">+ stasis_publish(topic, msg);</span><br><span style="color: hsl(120, 100%, 40%);">+ ao2_cleanup(msg);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ return 0;</span><br><span style="color: hsl(120, 100%, 40%);">+}</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> static void cpg_confchg_cb(cpg_handle_t handle, const struct cpg_name *group_name,</span><br><span> const struct cpg_address *member_list, size_t member_list_entries,</span><br><span> const struct cpg_address *left_list, size_t left_list_entries,</span><br><span>@@ -602,12 +655,43 @@</span><br><span> for (i = 0; i < left_list_entries; i++) {</span><br><span> const struct cpg_address *cpg_node = &left_list[i];</span><br><span> struct corosync_node* node;</span><br><span style="color: hsl(120, 100%, 40%);">+ unsigned int j;</span><br><span> </span><br><span> node = ao2_find(nodes, &cpg_node->nodeid, OBJ_UNLINK | OBJ_SEARCH_KEY);</span><br><span> if (!node) {</span><br><span> continue;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ for (j = 0; j < ARRAY_LEN(event_types); j++) {</span><br><span style="color: hsl(120, 100%, 40%);">+ struct ao2_container *messages;</span><br><span style="color: hsl(120, 100%, 40%);">+ int messages_count;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_rdlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb rdlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!event_types[j].subscribe) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!event_types[j].cache_fn || !event_types[j].message_type_fn) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ messages = stasis_cache_dump_by_eid(event_types[j].cache_fn(), event_types[j].message_type_fn(), &node->eid);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ messages_count = ao2_container_count(messages);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Clearing %i events of type %s of node %i from stasis cache.\n", messages_count, event_types[j].name, node->id);</span><br><span style="color: hsl(120, 100%, 40%);">+ ao2_callback(messages, OBJ_NODATA, clear_node_cache, event_types[j].topic_fn());</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Cleared events of type %s from stasis cache.\n", event_types[j].name);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ao2_t_ref(messages, -1, "Dispose of flushed cache");</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> publish_cluster_discovery_to_stasis_full(node, 0);</span><br><span> ao2_ref(node, -1);</span><br><span> }</span><br><span>@@ -620,24 +704,30 @@</span><br><span> </span><br><span> for (i = 0; i < ARRAY_LEN(event_types); i++) {</span><br><span> struct ao2_container *messages;</span><br><span style="color: hsl(120, 100%, 40%);">+ int messages_count;</span><br><span> </span><br><span> ast_rwlock_rdlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb rdlock\n");</span><br><span> if (!event_types[i].publish) {</span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb unlock\n");</span><br><span> continue;</span><br><span> }</span><br><span> </span><br><span> if (!event_types[i].cache_fn || !event_types[i].message_type_fn) {</span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb unlock\n");</span><br><span> continue;</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- messages = stasis_cache_dump_by_eid(event_types[i].cache_fn(),</span><br><span style="color: hsl(0, 100%, 40%);">- event_types[i].message_type_fn(),</span><br><span style="color: hsl(0, 100%, 40%);">- &ast_eid_default);</span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cpg_confchg_cb unlock\n");</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ messages = stasis_cache_dump_by_eid(event_types[i].cache_fn(), event_types[i].message_type_fn(), &ast_eid_default);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ messages_count = ao2_container_count(messages);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Sending %i events of type %s to corosync.\n", messages_count, event_types[i].name);</span><br><span> ao2_callback(messages, OBJ_NODATA, dump_cache_cb, NULL);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Sent events of type %s to corosync.\n", event_types[i].name);</span><br><span> </span><br><span> ao2_t_ref(messages, -1, "Dispose of dumped cache");</span><br><span> }</span><br><span>@@ -656,14 +746,21 @@</span><br><span> char buf[128];</span><br><span> int res;</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = corosync_cfg_local_get(cfg_handle, &node_id)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_WARNING, "Failed to extract Corosync node ID for this node. Not informing cluster of existance.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- return;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_tryrdlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "send_cluster_notify rdlock\n");</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (((cs_err = corosync_cfg_get_node_addrs(cfg_handle, node_id, 1, &num_addrs, &corosync_addr)) != CS_OK) || (num_addrs < 1)) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_WARNING, "Failed to get local Corosync address. Not informing cluster of existance.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- return;</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = corosync_cfg_local_get(cfg_handle, &node_id)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to extract Corosync node ID for this node. Not informing cluster of existance.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ return;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if (((cs_err = corosync_cfg_get_node_addrs(cfg_handle, node_id, 1, &num_addrs, &corosync_addr)) != CS_OK) || (num_addrs < 1)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to get local Corosync address. Not informing cluster of existance.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ return;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "send_cluster_notify unlock\n");</span><br><span> }</span><br><span> </span><br><span> sa = (struct sockaddr *)corosync_addr.address;</span><br><span>@@ -679,7 +776,7 @@</span><br><span> AST_EVENT_IE_LOCAL_ADDR, AST_EVENT_IE_PLTYPE_STR, buf,</span><br><span> AST_EVENT_IE_END);</span><br><span> publish_event_to_corosync(event);</span><br><span style="color: hsl(0, 100%, 40%);">- ast_free(event);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_event_destroy(event);</span><br><span> }</span><br><span> </span><br><span> static void *dispatch_thread_handler(void *data)</span><br><span>@@ -691,18 +788,29 @@</span><br><span> { .events = POLLIN, },</span><br><span> };</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_fd_get(cpg_handle, &pfd[0].fd)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to get CPG fd. This module is now broken.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_tryrdlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler rdlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_fd_get(cpg_handle, &pfd[0].fd)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to get CPG fd. This module is now broken.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ return NULL;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = corosync_cfg_fd_get(cfg_handle, &pfd[1].fd)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to get CFG fd. This module is now broken.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ return NULL;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ pfd[2].fd = dispatch_thread.alert_pipe[0];</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to get fd: initiliazing CPG. This module is now broken.\n");</span><br><span> return NULL;</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = corosync_cfg_fd_get(cfg_handle, &pfd[1].fd)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to get CFG fd. This module is now broken.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- return NULL;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- pfd[2].fd = dispatch_thread.alert_pipe[0];</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> send_cluster_notify();</span><br><span> while (!dispatch_thread.stop) {</span><br><span> int res;</span><br><span>@@ -713,65 +821,141 @@</span><br><span> pfd[1].revents = 0;</span><br><span> pfd[2].revents = 0;</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- res = ast_poll(pfd, ARRAY_LEN(pfd), -1);</span><br><span style="color: hsl(120, 100%, 40%);">+ res = ast_poll(pfd, ARRAY_LEN(pfd), COROSYNC_POLL_TIMEOUT);</span><br><span> if (res == -1 && errno != EINTR && errno != EAGAIN) {</span><br><span> ast_log(LOG_ERROR, "poll() error: %s (%d)\n", strerror(errno), errno);</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = CS_ERR_BAD_HANDLE;</span><br><span style="color: hsl(120, 100%, 40%);">+ } else if (res == 0) {</span><br><span style="color: hsl(120, 100%, 40%);">+ unsigned int local_nodeid;</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (pfd[0].revents & POLLIN) {</span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_dispatch(cpg_handle, CS_DISPATCH_ALL)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_WARNING, "Failed CPG dispatch: %u\n", cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_tryrdlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler rdlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_local_get(cpg_handle, &local_nodeid)) == CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ struct cpg_name name;</span><br><span style="color: hsl(120, 100%, 40%);">+ struct cpg_address address[CPG_MEMBERS_MAX];</span><br><span style="color: hsl(120, 100%, 40%);">+ int entries = CPG_MEMBERS_MAX;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_copy_string(name.value, "asterisk", sizeof(name.value));</span><br><span style="color: hsl(120, 100%, 40%);">+ name.length = strlen(name.value);</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_membership_get(cpg_handle, &name, address, &entries)) == CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ int i;</span><br><span style="color: hsl(120, 100%, 40%);">+ int found = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(1, "CPG group has %i node membership\n", entries);</span><br><span style="color: hsl(120, 100%, 40%);">+ for (i = 0; (i < entries) && !found; i++) {</span><br><span style="color: hsl(120, 100%, 40%);">+ if (address[i].nodeid == local_nodeid)</span><br><span style="color: hsl(120, 100%, 40%);">+ found = 1;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!found) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to check CPG node membership\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = CS_ERR_BAD_HANDLE;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to get CPG node membership: %u\n", cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = CS_ERR_BAD_HANDLE;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to get CPG local node id: %u\n", cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = CS_ERR_BAD_HANDLE;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to check CPG node membership: initializing CPG.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = CS_ERR_BAD_HANDLE;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_tryrdlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler rdlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if (pfd[0].revents & POLLIN) {</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_dispatch(cpg_handle, CS_DISPATCH_ALL)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed CPG dispatch: %u\n", cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if (pfd[1].revents & POLLIN) {</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = corosync_cfg_dispatch(cfg_handle, CS_DISPATCH_ALL)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed CFG dispatch: %u\n", cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to dispatch: initializing CPG.\n");</span><br><span> }</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- if (pfd[1].revents & POLLIN) {</span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = corosync_cfg_dispatch(cfg_handle, CS_DISPATCH_ALL)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_WARNING, "Failed CFG dispatch: %u\n", cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> if (cs_err == CS_ERR_LIBRARY || cs_err == CS_ERR_BAD_HANDLE) {</span><br><span style="color: hsl(0, 100%, 40%);">- struct cpg_name name;</span><br><span> </span><br><span> /* If corosync gets restarted out from under Asterisk, try to recover. */</span><br><span> </span><br><span> ast_log(LOG_NOTICE, "Attempting to recover from corosync failure.\n");</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = corosync_cfg_initialize(&cfg_handle, &cfg_callbacks)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to initialize cfg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- sleep(5);</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_trywrlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ struct cpg_name name;</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler wrlock\n");</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_initialize(&cpg_handle, &cpg_callbacks) != CS_OK)) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to initialize cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- sleep(5);</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ if (cpg_handle && (cs_err = cpg_finalize(cpg_handle)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to finalize cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_fd_get(cpg_handle, &pfd[0].fd)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to get CPG fd.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- sleep(5);</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ if (cfg_handle && (cs_err = corosync_cfg_finalize(cfg_handle)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to finalize cfg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = corosync_cfg_fd_get(cfg_handle, &pfd[1].fd)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to get CFG fd.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- sleep(5);</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = corosync_cfg_initialize(&cfg_handle, &cfg_callbacks)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to initialize cfg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ sleep(5);</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- ast_copy_string(name.value, "asterisk", sizeof(name.value));</span><br><span style="color: hsl(0, 100%, 40%);">- name.length = strlen(name.value);</span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_join(cpg_handle, &name)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to join cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- sleep(5);</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_initialize(&cpg_handle, &cpg_callbacks) != CS_OK)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to initialize cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ sleep(5);</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_NOTICE, "Corosync recovery complete.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- send_cluster_notify();</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_fd_get(cpg_handle, &pfd[0].fd)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to get CPG fd.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ sleep(5);</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = corosync_cfg_fd_get(cfg_handle, &pfd[1].fd)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to get CFG fd.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ sleep(5);</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_copy_string(name.value, "asterisk", sizeof(name.value));</span><br><span style="color: hsl(120, 100%, 40%);">+ name.length = strlen(name.value);</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_join(cpg_handle, &name)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to join cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ sleep(5);</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 1;</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "dispatch_thread_handler unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Corosync recovery complete.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ send_cluster_notify();</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Failed to recover from corosync failure: initializing CPG.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> }</span><br><span> }</span><br><span> </span><br><span>@@ -801,64 +985,74 @@</span><br><span> return CLI_SHOWUSAGE;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- cs_err = cpg_iteration_initialize(cpg_handle, CPG_ITERATION_ALL, NULL, &cpg_iter);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_tryrdlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "corosync_show_members rdlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = cpg_iteration_initialize(cpg_handle, CPG_ITERATION_ALL, NULL, &cpg_iter);</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (cs_err != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "Failed to initialize CPG iterator.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- return CLI_FAILURE;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "\n"</span><br><span style="color: hsl(0, 100%, 40%);">- "=============================================================\n"</span><br><span style="color: hsl(0, 100%, 40%);">- "=== Cluster members =========================================\n"</span><br><span style="color: hsl(0, 100%, 40%);">- "=============================================================\n"</span><br><span style="color: hsl(0, 100%, 40%);">- "===\n");</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- for (i = 1, cs_err = cpg_iteration_next(cpg_iter, &cpg_desc);</span><br><span style="color: hsl(0, 100%, 40%);">- cs_err == CS_OK;</span><br><span style="color: hsl(0, 100%, 40%);">- cs_err = cpg_iteration_next(cpg_iter, &cpg_desc), i++) {</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef HAVE_COROSYNC_CFG_STATE_TRACK</span><br><span style="color: hsl(0, 100%, 40%);">- corosync_cfg_node_address_t addrs[8];</span><br><span style="color: hsl(0, 100%, 40%);">- int num_addrs = 0;</span><br><span style="color: hsl(0, 100%, 40%);">- unsigned int j;</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "=== Node %u\n", i);</span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "=== --> Group: %s\n", cpg_desc.group.value);</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef HAVE_COROSYNC_CFG_STATE_TRACK</span><br><span style="color: hsl(0, 100%, 40%);">- /*</span><br><span style="color: hsl(0, 100%, 40%);">- * Corosync 2.x cfg lib needs to allocate 1M on stack after calling</span><br><span style="color: hsl(0, 100%, 40%);">- * corosync_cfg_get_node_addrs. netconsole thread has allocated only 0.5M</span><br><span style="color: hsl(0, 100%, 40%);">- * resulting in crash.</span><br><span style="color: hsl(0, 100%, 40%);">- */</span><br><span style="color: hsl(0, 100%, 40%);">- cs_err = corosync_cfg_get_node_addrs(cfg_handle, cpg_desc.nodeid,</span><br><span style="color: hsl(0, 100%, 40%);">- ARRAY_LEN(addrs), &num_addrs, addrs);</span><br><span> if (cs_err != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_WARNING, "Failed to get node addresses\n");</span><br><span style="color: hsl(0, 100%, 40%);">- continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "Failed to initialize CPG iterator: %u.\n", cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ cpg_iteration_finalize(cpg_iter);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "corosync_show_members unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ return CLI_FAILURE;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- for (j = 0; j < num_addrs; j++) {</span><br><span style="color: hsl(0, 100%, 40%);">- struct sockaddr *sa = (struct sockaddr *) addrs[j].address;</span><br><span style="color: hsl(0, 100%, 40%);">- size_t sa_len = (size_t) addrs[j].address_length;</span><br><span style="color: hsl(0, 100%, 40%);">- char buf[128];</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "\n"</span><br><span style="color: hsl(120, 100%, 40%);">+ "=============================================================\n"</span><br><span style="color: hsl(120, 100%, 40%);">+ "=== Cluster members =========================================\n"</span><br><span style="color: hsl(120, 100%, 40%);">+ "=============================================================\n"</span><br><span style="color: hsl(120, 100%, 40%);">+ "===\n");</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- getnameinfo(sa, sa_len, buf, sizeof(buf), NULL, 0, NI_NUMERICHOST);</span><br><span style="color: hsl(120, 100%, 40%);">+ for (i = 1, cs_err = cpg_iteration_next(cpg_iter, &cpg_desc);</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err == CS_OK;</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = cpg_iteration_next(cpg_iter, &cpg_desc), i++) {</span><br><span style="color: hsl(120, 100%, 40%);">+ #ifdef HAVE_COROSYNC_CFG_STATE_TRACK</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_cfg_node_address_t addrs[8];</span><br><span style="color: hsl(120, 100%, 40%);">+ int num_addrs = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ unsigned int j;</span><br><span style="color: hsl(120, 100%, 40%);">+ #endif</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "=== --> Address %u: %s\n", j + 1, buf);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "=== Node %u\n", i);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "=== --> Group: %s\n", cpg_desc.group.value);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ #ifdef HAVE_COROSYNC_CFG_STATE_TRACK</span><br><span style="color: hsl(120, 100%, 40%);">+ /*</span><br><span style="color: hsl(120, 100%, 40%);">+ * Corosync 2.x cfg lib needs to allocate 1M on stack after calling</span><br><span style="color: hsl(120, 100%, 40%);">+ * corosync_cfg_get_node_addrs. netconsole thread has allocated only 0.5M</span><br><span style="color: hsl(120, 100%, 40%);">+ * resulting in crash.</span><br><span style="color: hsl(120, 100%, 40%);">+ */</span><br><span style="color: hsl(120, 100%, 40%);">+ cs_err = corosync_cfg_get_node_addrs(cfg_handle, cpg_desc.nodeid,</span><br><span style="color: hsl(120, 100%, 40%);">+ ARRAY_LEN(addrs), &num_addrs, addrs);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (cs_err != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_WARNING, "Failed to get node addresses\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ continue;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for (j = 0; j < num_addrs; j++) {</span><br><span style="color: hsl(120, 100%, 40%);">+ struct sockaddr *sa = (struct sockaddr *) addrs[j].address;</span><br><span style="color: hsl(120, 100%, 40%);">+ size_t sa_len = (size_t) addrs[j].address_length;</span><br><span style="color: hsl(120, 100%, 40%);">+ char buf[128];</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ getnameinfo(sa, sa_len, buf, sizeof(buf), NULL, 0, NI_NUMERICHOST);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "=== --> Address %u: %s\n", j + 1, buf);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ #else</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "=== --> Nodeid: %"PRIu32"\n", cpg_desc.nodeid);</span><br><span style="color: hsl(120, 100%, 40%);">+ #endif</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">-#else</span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "=== --> Nodeid: %"PRIu32"\n", cpg_desc.nodeid);</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "===\n"</span><br><span style="color: hsl(120, 100%, 40%);">+ "=============================================================\n"</span><br><span style="color: hsl(120, 100%, 40%);">+ "\n");</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ cpg_iteration_finalize(cpg_iter);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "corosync_show_members unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli(a->fd, "Failed to initialize CPG iterator: initializing CPG.\n");</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli(a->fd, "===\n"</span><br><span style="color: hsl(0, 100%, 40%);">- "=============================================================\n"</span><br><span style="color: hsl(0, 100%, 40%);">- "\n");</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- cpg_iteration_finalize(cpg_iter);</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> return CLI_SUCCESS;</span><br><span> }</span><br><span> </span><br><span>@@ -891,10 +1085,9 @@</span><br><span> return CLI_FAILURE;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- ast_rwlock_rdlock(&event_types_lock);</span><br><span> event_types[AST_EVENT_PING].publish_to_stasis(event);</span><br><span style="color: hsl(0, 100%, 40%);">- ast_rwlock_unlock(&event_types_lock);</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ ast_event_destroy(event);</span><br><span> return CLI_SUCCESS;</span><br><span> }</span><br><span> </span><br><span>@@ -925,6 +1118,7 @@</span><br><span> "===\n");</span><br><span> </span><br><span> ast_rwlock_rdlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "corosync_show_config rdlock\n");</span><br><span> for (i = 0; i < ARRAY_LEN(event_types); i++) {</span><br><span> if (event_types[i].publish) {</span><br><span> ast_cli(a->fd, "=== ==> Publishing Event Type: %s\n",</span><br><span>@@ -936,6 +1130,7 @@</span><br><span> }</span><br><span> }</span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "corosync_show_config unlock\n");</span><br><span> </span><br><span> ast_cli(a->fd, "===\n"</span><br><span> "=============================================================\n"</span><br><span>@@ -986,6 +1181,7 @@</span><br><span> unsigned int i;</span><br><span> </span><br><span> ast_rwlock_wrlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_general_config wrlock\n");</span><br><span> </span><br><span> for (i = 0; i < ARRAY_LEN(event_types); i++) {</span><br><span> event_types[i].publish = event_types[i].publish_default;</span><br><span>@@ -1018,6 +1214,7 @@</span><br><span> }</span><br><span> </span><br><span> ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_general_config unlock\n");</span><br><span> </span><br><span> return res;</span><br><span> }</span><br><span>@@ -1057,17 +1254,33 @@</span><br><span> if (stasis_router) {</span><br><span> </span><br><span> /* Unsubscribe all topic forwards and cancel all message routes */</span><br><span style="color: hsl(0, 100%, 40%);">- ast_rwlock_wrlock(&event_types_lock);</span><br><span> for (i = 0; i < ARRAY_LEN(event_types); i++) {</span><br><span style="color: hsl(120, 100%, 40%);">+ struct ao2_container *messages = NULL;</span><br><span style="color: hsl(120, 100%, 40%);">+ int messages_count;</span><br><span style="color: hsl(120, 100%, 40%);">+ unsigned char subscribe = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_wrlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cleanup_module wrlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ subscribe = event_types[i].subscribe;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> if (event_types[i].sub) {</span><br><span> event_types[i].sub = stasis_forward_cancel(event_types[i].sub);</span><br><span style="color: hsl(0, 100%, 40%);">- stasis_message_router_remove(stasis_router,</span><br><span style="color: hsl(0, 100%, 40%);">- event_types[i].message_type_fn());</span><br><span style="color: hsl(120, 100%, 40%);">+ stasis_message_router_remove(stasis_router, event_types[i].message_type_fn());</span><br><span> }</span><br><span> event_types[i].publish = 0;</span><br><span> event_types[i].subscribe = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&event_types_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cleanup_module unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if (subscribe && event_types[i].cache_fn && event_types[i].message_type_fn) {</span><br><span style="color: hsl(120, 100%, 40%);">+ messages = stasis_cache_dump_all(event_types[i].cache_fn(), event_types[i].message_type_fn());</span><br><span style="color: hsl(120, 100%, 40%);">+ messages_count = ao2_container_count(messages);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Clearing %i events of type %s of other nodes from stasis cache.\n", messages_count, event_types[i].name);</span><br><span style="color: hsl(120, 100%, 40%);">+ ao2_callback(messages, OBJ_NODATA, clear_node_cache, event_types[i].topic_fn());</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_NOTICE, "Cleared events of type %s from stasis cache.\n", event_types[i].name);</span><br><span style="color: hsl(120, 100%, 40%);">+ ao2_t_ref(messages, -1, "Dispose of flushed cache");</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">- ast_rwlock_unlock(&event_types_lock);</span><br><span> </span><br><span> stasis_message_router_unsubscribe_and_join(stasis_router);</span><br><span> stasis_router = NULL;</span><br><span>@@ -1101,16 +1314,21 @@</span><br><span> dispatch_thread.alert_pipe[1] = -1;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (cpg_handle && (cs_err = cpg_finalize(cpg_handle)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to finalize cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">- cpg_handle = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_trywrlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cleanup_module wrlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if (cpg_handle && (cs_err = cpg_finalize(cpg_handle)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to finalize cpg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ cpg_handle = 0;</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if (cfg_handle && (cs_err = corosync_cfg_finalize(cfg_handle)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to finalize cfg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (cfg_handle && (cs_err = corosync_cfg_finalize(cfg_handle)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to finalize cfg (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ cfg_handle = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "cleanup_module unlock\n");</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">- cfg_handle = 0;</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> ao2_cleanup(nodes);</span><br><span> nodes = NULL;</span><br><span> }</span><br><span>@@ -1142,6 +1360,8 @@</span><br><span> ast_log(AST_LOG_ERROR, "Failed to create message router for corosync topic\n");</span><br><span> goto failed;</span><br><span> }</span><br><span style="color: hsl(120, 100%, 40%);">+ stasis_message_router_set_congestion_limits(stasis_router, -1,</span><br><span style="color: hsl(120, 100%, 40%);">+ 10 * AST_TASKPROCESSOR_HIGH_WATER_LEVEL);</span><br><span> </span><br><span> if (STASIS_MESSAGE_TYPE_INIT(corosync_ping_message_type) != 0) {</span><br><span> ast_log(AST_LOG_ERROR, "Failed to initialize corosync ping message type\n");</span><br><span>@@ -1153,39 +1373,55 @@</span><br><span> goto failed;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = corosync_cfg_initialize(&cfg_handle, &cfg_callbacks)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to initialize cfg: (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (!ast_rwlock_trywrlock(&init_cpg_lock)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 0;</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_module wrlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = corosync_cfg_initialize(&cfg_handle, &cfg_callbacks)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to initialize cfg: (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_module unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ goto failed;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_initialize(&cpg_handle, &cpg_callbacks)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to initialize cpg: (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_module unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ goto failed;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_copy_string(name.value, "asterisk", sizeof(name.value));</span><br><span style="color: hsl(120, 100%, 40%);">+ name.length = strlen(name.value);</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if ((cs_err = cpg_join(cpg_handle, &name)) != CS_OK) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to join: (%d)\n", (int) cs_err);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_module unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ goto failed;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if (pipe(dispatch_thread.alert_pipe) == -1) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Failed to create alert pipe: %s (%d)\n",</span><br><span style="color: hsl(120, 100%, 40%);">+ strerror(errno), errno);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_module unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ goto failed;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+ corosync_node_joined = 1;</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_rwlock_unlock(&init_cpg_lock);</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_debug(5, "load_module unlock\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ if (corosync_pthread_create_background(&dispatch_thread.id, NULL,</span><br><span style="color: hsl(120, 100%, 40%);">+ dispatch_thread_handler, NULL)) {</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_log(LOG_ERROR, "Error starting CPG dispatch thread.\n");</span><br><span style="color: hsl(120, 100%, 40%);">+ goto failed;</span><br><span style="color: hsl(120, 100%, 40%);">+ }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ast_cli_register_multiple(corosync_cli, ARRAY_LEN(corosync_cli));</span><br><span style="color: hsl(120, 100%, 40%);">+ } else {</span><br><span> goto failed;</span><br><span> }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_initialize(&cpg_handle, &cpg_callbacks)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to initialize cpg: (%d)\n", (int) cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- goto failed;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- ast_copy_string(name.value, "asterisk", sizeof(name.value));</span><br><span style="color: hsl(0, 100%, 40%);">- name.length = strlen(name.value);</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- if ((cs_err = cpg_join(cpg_handle, &name)) != CS_OK) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to join: (%d)\n", (int) cs_err);</span><br><span style="color: hsl(0, 100%, 40%);">- goto failed;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- if (pipe(dispatch_thread.alert_pipe) == -1) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Failed to create alert pipe: %s (%d)\n",</span><br><span style="color: hsl(0, 100%, 40%);">- strerror(errno), errno);</span><br><span style="color: hsl(0, 100%, 40%);">- goto failed;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- if (corosync_pthread_create_background(&dispatch_thread.id, NULL,</span><br><span style="color: hsl(0, 100%, 40%);">- dispatch_thread_handler, NULL)) {</span><br><span style="color: hsl(0, 100%, 40%);">- ast_log(LOG_ERROR, "Error starting CPG dispatch thread.\n");</span><br><span style="color: hsl(0, 100%, 40%);">- goto failed;</span><br><span style="color: hsl(0, 100%, 40%);">- }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- ast_cli_register_multiple(corosync_cli, ARRAY_LEN(corosync_cli));</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> return AST_MODULE_LOAD_SUCCESS;</span><br><span> </span><br><span> failed:</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.asterisk.org/c/asterisk/+/14603">change 14603</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.asterisk.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.asterisk.org/c/asterisk/+/14603"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: asterisk </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: Ie8e99bc23f141a73c13ae6fb1948d148d4de17f2 </div>
<div style="display:none"> Gerrit-Change-Number: 14603 </div>
<div style="display:none"> Gerrit-PatchSet: 2 </div>
<div style="display:none"> Gerrit-Owner: George Joseph <gjoseph@digium.com> </div>
<div style="display:none"> Gerrit-Reviewer: Friendly Automation </div>
<div style="display:none"> Gerrit-Reviewer: Joshua Colp <jcolp@sangoma.com> </div>
<div style="display:none"> Gerrit-Reviewer: Kevin Harwell <kharwell@digium.com> </div>
<div style="display:none"> Gerrit-Reviewer: Università di Bologna - CESIA VoIP <cesia.voip@unibo.it> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>