[Asterisk-code-review] taskprocessor: Add high water mark warnings (asterisk[certified/13.1])

[Jonathan Rose]

Jonathan Rose has uploaded a new change for review.

  https://gerrit.asterisk.org/1573

Change subject: taskprocessor: Add high water mark warnings
......................................................................

taskprocessor: Add high water mark warnings

If a taskprocessor's queue grows large, this can indicate that there
may be a problem with tasks not leaving the processor or else that
the number of available task processors for a given type of task is
too low. This patch makes it so that if a taskprocessor's task queue
grows above 100 queued tasks that it will emit a warning message.
Warning messages are emitted only once per task processor.

ASTERISK-25518 #close
Reported by: Jonathan Rose

Change-Id: Ib1607c35d18c1d6a0575b3f0e3ff5d932fd6600c
---
M main/taskprocessor.c
1 file changed, 11 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.asterisk.org:29418/asterisk refs/changes/73/1573/1

diff --git a/main/taskprocessor.c b/main/taskprocessor.c
index 8a078f1..227fc3e 100644
--- a/main/taskprocessor.c
+++ b/main/taskprocessor.c
@@ -83,6 +83,8 @@
 	pthread_t thread;
 	/*! Indicates if the taskprocessor is currently executing a task */
 	unsigned int executing:1;
+	/*! Indicates that a high water warning has been issued on this task processor */
+	unsigned int high_water_warned:1;
 };
 
 /*!
@@ -715,6 +717,8 @@
 	return NULL;
 }
 
+#define HIGH_WATER_LEVEL 100
+
 /* push the task into the taskprocessor queue */
 static int taskprocessor_push(struct ast_taskprocessor *tps, struct tps_task *t)
 {
@@ -734,6 +738,13 @@
 	ao2_lock(tps);
 	AST_LIST_INSERT_TAIL(&tps->tps_queue, t, list);
 	previous_size = tps->tps_queue_size++;
+
+	if (previous_size >= HIGH_WATER_LEVEL && !tps->high_water_warned) {
+		ast_log(LOG_WARNING, "The '%s' task processor queue reached %d scheduled tasks.\n",
+			tps->name, previous_size);
+		tps->high_water_warned = 1;
+	}
+
 	/* The currently executing task counts as still in queue */
 	was_empty = tps->executing ? 0 : previous_size == 0;
 	ao2_unlock(tps);

-- 
To view, visit https://gerrit.asterisk.org/1573
To unsubscribe, visit https://gerrit.asterisk.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib1607c35d18c1d6a0575b3f0e3ff5d932fd6600c
Gerrit-PatchSet: 1
Gerrit-Project: asterisk
Gerrit-Branch: certified/13.1
Gerrit-Owner: Jonathan Rose <jrose at digium.com>