<html>
<head>
<base href="https://wiki.asterisk.org/wiki">
<link rel="stylesheet" href="/wiki/s/2036/1/7/_/styles/combined.css?spaceKey=TOP&forWysiwyg=true" type="text/css">
</head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
<h2><a href="https://wiki.asterisk.org/wiki/display/TOP/Thread+Pools">Thread Pools</a></h2>
<h4>Page <b>edited</b> by <a href="https://wiki.asterisk.org/wiki/display/~mmichelson">Mark Michelson</a>
</h4>
<br/>
<h4>Changes (5)</h4>
<div id="page-diffs">
<table class="diff" cellpadding="0" cellspacing="0">
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" > <br>* Frequency of tasks being queued <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">* Backlog of tasks in each worker thread <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">* Number of tasks in threads <br></td></tr>
<tr><td class="diff-unchanged" >* Average time delta between a task's being queued and its execution (Turnaround time) <br> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">XXX Revisit this <br> <br></td></tr>
<tr><td class="diff-unchanged" >h5. Growth amount <br>Once it's been decided that the thread pool should grow, by how many threads should the pool grow? Potential methods are to grow by a fixed amount or to double the size of the thread pool. <br></td></tr>
<tr><td class="diff-snipped" >...<br></td></tr>
<tr><td class="diff-unchanged" >Both dynamic and static thread pools will need an initial size. The static thread pool will just remain that size forever whereas the dynamic thread pool will adjust based on other parameters. <br> <br></td></tr>
<tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">To be continued <br></td></tr>
<tr><td class="diff-added-lines" style="background-color: #dfd;">h1. The implementation <br> <br>With all the configuration considerations, here is a sample class for a thread pool. Liberties have been taken with regards to namespaces. <br> <br>{code: language=c++} <br>/** <br> * A base class for defining growth <br> * policies for a thread pool. <br> */ <br>class GrowthPolicy <br>{ <br>public: <br> // Determine if the growth policy declares that <br> // it is time for the thread pool to grow. The <br> // thread pool will call this method for each new <br> // task that is queued. The thread pool is provided <br> // in case it is needed in order to determine whether <br> // growth is necessary. <br> virtual bool shouldGrow(const ThreadPool &pool) = 0; <br> <br> // When the thread pool needs to grow, this method <br> // will be called to determine by how much the thread <br> // pool should grow. The thread pool is provided to <br> // this function in case the GrowthPolicy needs to <br> // gather statistics from it to determine its answer. <br> virtual size_t growthInterval(const ThreadPool &pool) = 0; <br> <br> // The thread pool will call this to determine which threads <br> // may be removed from the pool. <br> virtual std::vector<boost::shared_ptr<PoolId> > killableThreads(const ThreadPool &pool) = 0; <br>protected: <br> // You can't construct one of these. You must use a subclass. <br> GrowthPolicy(); <br>}; <br> <br>class ThreadPool : public WorkQueue <br>{ <br>public: <br> //Create thread pool with given growth policy and initial number of threads. <br> //If growthPolicy is 0, then the thread pool will be static, i.e. it will not grow or shrink <br> ThreadPool(const boost::shared_ptr<GrowthPolicy> growthPolicy, size_t initThreads); <br> <br> //Overriding the WorkQueue methods <br> virtual void enqueue(WorkPtr &w, PoolId id); <br> virtual boost::shared_ptr<PoolId> enqueue(WorkPtr &w); <br> <br>private: <br> <br> //Grow the thread pool by growthInterval threads. The thread pool can determine <br> //how many new threads to add by calling GrowthPolicy::growthInterval(*this) <br> void grow(size_t growthInterval); <br> <br> boost::shared_ptr<GrowthPolicy> mPolicy; <br> std::map<PoolId, boost::shared_ptr<SimpleWorkQueue> > mQueues; <br>}; <br>{code} <br> <br>So first, let's have a look at {{ThreadPool}}. It is a subclass of {{WorkQueue}}, meaning that it overrides the method to enqueue new tasks. At the heart of the {{ThreadPool}} is a map of {{SimpleWorkQueues}}. A map was chosen because it allows for easy access to a specific work queue if desired. <br></td></tr>
</table>
</div> <h4>Full Content</h4>
<div class="notificationGreySide">
<div class='panelMacro'><table class='warningMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/wiki/images/icons/emoticons/forbidden.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td>Under construction</td></tr></table></div>
<p>In <a href="/wiki/display/TOP/Component+Threading+Design+for+Asynchronous+Operations" title="Component Threading Design for Asynchronous Operations">Component Threading Design for Asynchronous Operations</a>, the <tt>Workqueue</tt> interface was defined, and an implementation, called a <tt>SimpleWorkQueue</tt> was proposed. While the <tt>SimpleWorkQueue</tt> is a good implementation for instances where a single thread is needed, another implementation is needed when multiple threads are required.</p>
<h1><a name="ThreadPools-Configurationconsiderations"></a>Configuration considerations</h1>
<p>Thread pools have a potential for lots of knobs to be tweaked regarding their operation.</p>
<h3><a name="ThreadPools-StaticorDynamic"></a>Static or Dynamic</h3>
<p>A static thread pool is one in which a set number of threads are created at the time that the thread pool is constructed. The thread pool will always have this number of worker threads no matter the amount of system load. Dynamic thread pools start with some initial number of threads and then can grow or shrink depending on the amount of load on the system. Since static thread pools are simple, we will focus on the dynamic thread pool configuration.</p>
<h3><a name="ThreadPools-GrowthPolicy"></a>Growth Policy</h3>
<p>It's obvious that a dynamic thread pool should grow when needed, but there needs to be some set of rules to govern the growth. </p>
<h5><a name="ThreadPools-MaxSize"></a>Max Size</h5>
<p>One valid concern is letting the thread pool grow to an unreasonable size, so an obvious configuration option for dynamic thread pools is a maximum size. Once the thread pool has reached its max capacity it will no longer grow, even if policy states that the load is such that it should grow more.</p>
<h5><a name="ThreadPools-GrowthConditions"></a>Growth Conditions</h5>
<p>What also needs to be configurable is the determination of when to increase the size of the thread pool. "Load" can be determined in several ways, including</p>
<ul>
<li>Frequency of tasks being queued</li>
<li>Number of tasks in threads</li>
<li>Average time delta between a task's being queued and its execution (Turnaround time)</li>
</ul>
<h5><a name="ThreadPools-Growthamount"></a>Growth amount</h5>
<p>Once it's been decided that the thread pool should grow, by how many threads should the pool grow? Potential methods are to grow by a fixed amount or to double the size of the thread pool.</p>
<h3><a name="ThreadPools-ShrinkingPolicy"></a>Shrinking Policy</h3>
<p>A dynamic thread pool may grow as needed, but should the thread pool shrink if the system load has decreased? It certainly seems like it would be desired. Otherwise a spike in load could cause a permanent allocation of threads which would barely ever be used.</p>
<h5><a name="ThreadPools-MinSize"></a>Min Size</h5>
<p>It's useful to be able to shrink the thread pool, but it also seems prudent to make sure the thread pool doesn't get <b>too</b> small. Specifying a minimum size will allow for the thread pool to shrink only to a specified minimum.</p>
<h5><a name="ThreadPools-ShrinkConditions"></a>Shrink Conditions</h5>
<p>Like with growth, it needs to be determined what metric should be used to determine when a thread pool may shrink. Determining inactivity in a thread can really only be done by measuring the time that a thread has been idle. After a certain cutoff point, the thread can be considered "dead" and be removed from the pool. Unlike with thread pool growth, you can't really have a configured shrink amount since the pool must shrink a single thread at a time.</p>
<h3><a name="ThreadPools-Initialsize"></a>Initial size</h3>
<p>Both dynamic and static thread pools will need an initial size. The static thread pool will just remain that size forever whereas the dynamic thread pool will adjust based on other parameters.</p>
<h1><a name="ThreadPools-Theimplementation"></a>The implementation</h1>
<p>With all the configuration considerations, here is a sample class for a thread pool. Liberties have been taken with regards to namespaces.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<script type="syntaxhighlighter" class="toolbar: false; theme: Confluence; brush: java; gutter: false"><![CDATA[
/**
* A base class for defining growth
* policies for a thread pool.
*/
class GrowthPolicy
{
public:
// Determine if the growth policy declares that
// it is time for the thread pool to grow. The
// thread pool will call this method for each new
// task that is queued. The thread pool is provided
// in case it is needed in order to determine whether
// growth is necessary.
virtual bool shouldGrow(const ThreadPool &pool) = 0;
// When the thread pool needs to grow, this method
// will be called to determine by how much the thread
// pool should grow. The thread pool is provided to
// this function in case the GrowthPolicy needs to
// gather statistics from it to determine its answer.
virtual size_t growthInterval(const ThreadPool &pool) = 0;
// The thread pool will call this to determine which threads
// may be removed from the pool.
virtual std::vector<boost::shared_ptr<PoolId> > killableThreads(const ThreadPool &pool) = 0;
protected:
// You can't construct one of these. You must use a subclass.
GrowthPolicy();
};
class ThreadPool : public WorkQueue
{
public:
//Create thread pool with given growth policy and initial number of threads.
//If growthPolicy is 0, then the thread pool will be static, i.e. it will not grow or shrink
ThreadPool(const boost::shared_ptr<GrowthPolicy> growthPolicy, size_t initThreads);
//Overriding the WorkQueue methods
virtual void enqueue(WorkPtr &w, PoolId id);
virtual boost::shared_ptr<PoolId> enqueue(WorkPtr &w);
private:
//Grow the thread pool by growthInterval threads. The thread pool can determine
//how many new threads to add by calling GrowthPolicy::growthInterval(*this)
void grow(size_t growthInterval);
boost::shared_ptr<GrowthPolicy> mPolicy;
std::map<PoolId, boost::shared_ptr<SimpleWorkQueue> > mQueues;
};
]]></script>
</div></div>
<p>So first, let's have a look at <tt>ThreadPool</tt>. It is a subclass of <tt>WorkQueue</tt>, meaning that it overrides the method to enqueue new tasks. At the heart of the <tt>ThreadPool</tt> is a map of <tt>SimpleWorkQueues</tt>. A map was chosen because it allows for easy access to a specific work queue if desired.</p>
</div>
<div id="commentsSection" class="wiki-content pageSection">
<div style="float: right;">
<a href="https://wiki.asterisk.org/wiki/users/viewnotifications.action" class="grey">Change Notification Preferences</a>
</div>
<a href="https://wiki.asterisk.org/wiki/display/TOP/Thread+Pools">View Online</a>
|
<a href="https://wiki.asterisk.org/wiki/pages/diffpagesbyversion.action?pageId=11338403&revisedVersion=2&originalVersion=1">View Changes</a>
|
<a href="https://wiki.asterisk.org/wiki/display/TOP/Thread+Pools?showComments=true&showCommentArea=true#addcomment">Add Comment</a>
</div>
</div>
</div>
</div>
</div>
</body>
</html>