[Asterisk-code-review] taskprocessor: Add high water mark warnings (asterisk[master])

[Joshua Colp]

Joshua Colp has submitted this change and it was merged.

Change subject: taskprocessor: Add high water mark warnings
......................................................................


taskprocessor: Add high water mark warnings

If a taskprocessor's queue grows large, this can indicate that there
may be a problem with tasks not leaving the processor or else that
the number of available task processors for a given type of task is
too low. This patch makes it so that if a taskprocessor's task queue
grows above 100 queued tasks that it will emit a warning message.
Warning messages are emitted only once per task processor.

ASTERISK-25518 #close
Reported by: Jonathan Rose

Change-Id: Ib1607c35d18c1d6a0575b3f0e3ff5d932fd6600c
---
M main/taskprocessor.c
1 file changed, 11 insertions(+), 0 deletions(-)

Approvals:
  Mark Michelson: Looks good to me, but someone else must approve
  Richard Mudgett: Looks good to me, but someone else must approve
  Anonymous Coward #1000019: Verified
  Joshua Colp: Looks good to me, approved



diff --git a/main/taskprocessor.c b/main/taskprocessor.c
index 5c513ee..0719dee 100644
--- a/main/taskprocessor.c
+++ b/main/taskprocessor.c
@@ -83,6 +83,8 @@
 	pthread_t thread;
 	/*! Indicates if the taskprocessor is currently executing a task */
 	unsigned int executing:1;
+	/*! Indicates that a high water warning has been issued on this task processor */
+	unsigned int high_water_warned:1;
 };
 
 /*!
@@ -714,6 +716,8 @@
 	return NULL;
 }
 
+#define HIGH_WATER_LEVEL 100
+
 /* push the task into the taskprocessor queue */
 static int taskprocessor_push(struct ast_taskprocessor *tps, struct tps_task *t)
 {
@@ -733,6 +737,13 @@
 	ao2_lock(tps);
 	AST_LIST_INSERT_TAIL(&tps->tps_queue, t, list);
 	previous_size = tps->tps_queue_size++;
+
+	if (previous_size >= HIGH_WATER_LEVEL && !tps->high_water_warned) {
+		ast_log(LOG_WARNING, "The '%s' task processor queue reached %d scheduled tasks.\n",
+			tps->name, previous_size);
+		tps->high_water_warned = 1;
+	}
+
 	/* The currently executing task counts as still in queue */
 	was_empty = tps->executing ? 0 : previous_size == 0;
 	ao2_unlock(tps);

-- 
To view, visit https://gerrit.asterisk.org/1562
To unsubscribe, visit https://gerrit.asterisk.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ib1607c35d18c1d6a0575b3f0e3ff5d932fd6600c
Gerrit-PatchSet: 4
Gerrit-Project: asterisk
Gerrit-Branch: master
Gerrit-Owner: Jonathan Rose <jrose at digium.com>
Gerrit-Reviewer: Anonymous Coward #1000019
Gerrit-Reviewer: Corey Farrell <git at cfware.com>
Gerrit-Reviewer: Jonathan Rose <jrose at digium.com>
Gerrit-Reviewer: Joshua Colp <jcolp at digium.com>
Gerrit-Reviewer: Mark Michelson <mmichelson at digium.com>
Gerrit-Reviewer: Matt Jordan <mjordan at digium.com>
Gerrit-Reviewer: Richard Mudgett <rmudgett at digium.com>