[asterisk-dev] [Code Review]: Threadpool Support

[David Lee]

> On Jan. 10, 2013, 1 p.m., David Lee wrote:
> > /trunk/main/threadpool.c, line 66
> > <https://reviewboard.asterisk.org/r/2242/diff/2/?file=32641#file32641line66>
> >
> >     I think rather than using a taskprocessor here, maybe what you really want to do is extract the shared threadpool/taskprocessor code into a third thing.
> 
> Mark Michelson wrote:
>     Can you elaborate a bit? "A third thing" is not very descriptive of what you are trying to convey.

Sure. By the time I wrote that, my brain was in a code-review induced stupor.

Basically, instead of having a default-behavior taskprocessor and a specialized-behavior taskprocessor, what if you extracted out a thread-safe queue that you can attach listeners to. You would reuse the queue for both taskprocessor and threadpool, without having different kinds of taskprocessors.

Coming at this fresh, though, I may be simultaneously overthinking (excessive object design) and underthinking (how would you do that in C?).


> On Jan. 10, 2013, 1 p.m., David Lee wrote:
> > /trunk/include/asterisk/taskprocessor.h, line 143
> > <https://reviewboard.asterisk.org/r/2242/diff/2/?file=32638#file32638line143>
> >
> >     I think I see what you're doing, and I think you're exposing too much through the API.
> >      * Make ast_taskprocessor_listener and opaque struct
> >        * add accessor functions if you need any
> >      * get rid of the private_data alloc and free callbacks; just put that data in the taskprocessor struct
> 
> Mark Michelson wrote:
>     I had gone down the road of making taskprocessor listener an opaque struct, but I didn't think it was really worth it when I originally coded this.
>     
>     I have to disagree with your suggestion of putting the private data in the taskprocessor struct. IMO, the taskprocessor and the listener are two separate entities, and in the case of the two taskprocessor listeners that exist now, the default listener and the threadpool, the private data is intrinsic to the listener, not the taskprocessor. For instance, the default taskprocessor listener runs a thread that executes the tasks in the taskprocessor. Information about the thread and its execution belongs in the listener, not the taskprocessor. I guess you're seeing it more that the listener is a part of the taskprocessor and so having the private data in the taskprocessor makes just as much sense as having it in the listener.
>     
>     I'd be willing to get rid of the alloc callback potentially. ast_taskprocessor_listener_alloc() could just take another parameter that is the private data. The reason it works the way it does right now is due to some chicken-and-egg logic that exists. The private data needs a reference to the taskprocessor so it can execute tasks, the taskprocessor listener owns the private data, and the taskprocessor has a reference to the taskprocessor listener. My solution was to add an allocation function so that listener private data could be allocated once everything else in the taskprocessor and its listener was set up. If I get rid of the alloc callback, then I can just let the private data get its reference to the taskprocessor callback in the listener's start() callback.

Yeah; I don't think I really understood what you were doing there.

I still think getting rid of the alloc/free callbacks would be a Good Thing(TM). Passing the private data pointer in the _alloc() call makes more sense to me.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/2242/#review7651
-----------------------------------------------------------


On Jan. 7, 2013, 4:20 p.m., Mark Michelson wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/2242/
> -----------------------------------------------------------
> 
> (Updated Jan. 7, 2013, 4:20 p.m.)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Summary
> -------
> 
> This review adds a generic threadpool for Asterisk.
> 
> The threadpool implementation here is very similar to the one implemented in Asterisk SCF. Since this is implemented in C, the "is a" semantics offered by C++ are not available, so that had to be worked around.
> 
> The threadpool, when created, creates a taskprocessor. The threadpool itself exists as the private data on a taskprocessor listener. In this way, the threadpool can react to changes on the taskprocessor by informing its threadpool listener.
> 
> The threadpool informs its listeners of various changes:
> - When a task gets pushed into the threadpool
> - When the threadpool's task list has become empty
> - When the state of the threadpool's threads has changed, such as when an active thread goes idle or an idle thread is destroyed.
> 
> The threadpool listener can react to these changes as it sees fit. This allows for different policies to be adopted by different modules.
> 
> The offers some options for automatic behavior for common forms of operation. At allocation, an idle timeout can be specified in order to allow for idle threads to automatically get removed from the threadpool once they have been idle for a certain amount of time. Also, an automatic increment can be specified if the threadpool has tasks added to it and no idle threads are available to handle the task. With these, it may be possible for listeners only to intervene in certain situations. More options can possibly be added if they are not too policy-specific.
> 
> This set of changes is dependent on the set of taskprocessor changes introduced in https://reviewboard.asterisk.org/r/2200 . Since these changes were developed in the same branch as the taskprocessor changes, it means that all the taskprocessor changes are also included in this review. While there are minor changes in the taskprocessor code here as compared to the code in review 2200, they are very minor changes, and so close scrutiny of the taskprocessor changes is not as important as the threadpool code itself.
> 
> 
> This addresses bug ASTERISK-20691.
>     https://issues.asterisk.org/jira/browse/ASTERISK-20691
> 
> 
> Diffs
> -----
> 
>   /trunk/main/threadpool.c PRE-CREATION 
>   /trunk/include/asterisk/taskprocessor.h 378146 
>   /trunk/include/asterisk/threadpool.h PRE-CREATION 
>   /trunk/main/taskprocessor.c 378146 
>   /trunk/tests/test_taskprocessor.c PRE-CREATION 
>   /trunk/tests/test_threadpool.c PRE-CREATION 
> 
> Diff: https://reviewboard.asterisk.org/r/2242/diff
> 
> 
> Testing
> -------
> 
> A suite of unit tests have been added to ensure that the threadpool works as expected. They all pass.
> 
> 
> Thanks,
> 
> Mark
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20130114/a21cd648/attachment.htm>