[asterisk-dev] [Code Review]: A new higher-level API for working with Asterisk configs, with example code in app_skel.c and udptl.c
Terry Wilson
reviewboard at asterisk.org
Thu Apr 26 16:33:37 CDT 2012
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/include/asterisk/config_options.h, lines 43-47
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27365#file27365line43>
> >
> > So I take it that the idea here is that you can specify that an aco_type should process any category in a configuration file whose name either matches or does not match a regex. So for instance, if you have a "general" section and then every other section is used to configure "foos" then the aco_type corresponding to a "foo" would have some sort of regex blacklist for "general".
> >
> > I don't think this is quite enough to be able to distinguish different types of configurable options. For instance, in iax.conf, you can't tell the difference between an iax2_user and an iax2_peer just based on the context name. You also have to look at the "type=" option to determine that.
Right. There are two (added at the last minute and completely undocumented! sorry!) fields in aco_type_private_alloc(), matchfield, and matchvalue, that would allow one to specify a type= kind of option to allow differentiation between multiple types. The ao2_callback to look up a type will use the matchfield/matchvalue when searching. There is still a problem, though, for IAX2 where type=friend actually creates a peer AND a user on the backend. I think, that in general, we should just discourage this kind of behavior. Honestly, iax2_user has about 7 fields that aren't also in iax2_peer, so merging them into a single type is very little overhead.
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/include/asterisk/config_options.h, lines 49-54
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27365#file27365line49>
> >
> > These really need more documentation to help understand what they would be useful for. Obviously they are hooks into the various stages of configuration, but names like "post_cfg_init" and "prelink" don't give much insight into when they are called or what they might be used for. Unlike the other hooks in this list, you don't have any examples of these being used.
Yeah, documentation is forthcoming. I just wanted to make sure that the general idea seemed sound to people before putting in a couple of more days doing extensive documentation. I should have had a little more set up for review purposes, though. By the time I got this part done, my brain was pretty fried.
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/include/asterisk/config_options.h, line 68
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27365#file27365line68>
> >
> > What happens if a configurable object uses data from multiple files? For instance, chan_sip loads both sip.conf and users.conf when creating SIP peers.
I'm not sure it would be possible to make a really high-level API that people would still want to use that would *automatically* handle merging data from both files before doing the load/reload, since matching what field in one file goes with the field in the other seems like it could be kind of hard depending on how things were set up. BUT, with that said, I think it would be possible to use the post_cfg_init callback, which basically gives you a hook for "after defaults are set, but before options are parsed" to call aco_process_category_options on a separate config file/category to pull in values from the other file. Then, afterwards, normal processing would occur. If you wanted to process the other file *after* the main file, then the prelink callback could be used to do the same thing.
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/include/asterisk/config_options.h, line 189
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27365#file27365line189>
> >
> > There is no struct type parameter in this macro.
Ooops, it was totally there like 2 different versions ago. :-p
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/include/asterisk/config_options.h, lines 247-250
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27365#file27365line247>
> >
> > What is "in" ?
"in" is basically an accumulator. So, we start out by passing in the number of args, then on the first pass of the macro to ARGMAP_${N}, we append the func(func_arg, ${argN}) to "in" giving us N, func(func_arg, ${arg}N) which we then put in "in" for the next call to ARGMAP_${N-1}, so we end up with "N, func(func_arg, ${arg1}), func(func_arg, ${arg2}), ..., ...func(func_arg, ${argN}) by the time all is done. It is...a little crazy.
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/main/config_options.c, lines 91-98
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27371#file27371line91>
> >
> > One type of function you're missing here is one to copy a string into a buffer (think CV_STR from config.h).
Indeed. I'll have to add that. Hopefully, as people go along, if they need to use a custom handler and it makes sense at all to do so, we can get them to write a default handler version of it.
> On April 26, 2012, 3:52 p.m., Mark Michelson wrote:
> > /trunk/main/udptl.c, lines 85-86
> > <https://reviewboard.asterisk.org/r/1873/diff/1/?file=27372#file27372line85>
> >
> > A similar macro called __stringify exists in optional_api.h. It seems like this sort of macro would be useful to have in utils.h or strings.h
Yeah. And the PASTE macro in config_options.h as well. I'll just move them both there--possibly with the names STRINGIFY and PASTE.
- Terry
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/1873/#review6070
-----------------------------------------------------------
On April 16, 2012, 1:45 p.m., Terry Wilson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/1873/
> -----------------------------------------------------------
>
> (Updated April 16, 2012, 1:45 p.m.)
>
>
> Review request for Asterisk Developers.
>
>
> Summary
> -------
>
> This review supersedes the ones at reviews 1840 and 1841. There is still a lot of cleanup/documentation work to do. This review is mostly about the overall idea/method. Doing the finishing work before someone comes up with why it is all a bad idea seems like...a bad idea. If you want to comment on non-big-picture stuff, feel free, but most important at this stage is whether or not this is a good idea at all.
>
> The goal of this patch is to come up with a way to safely handle the loading and reloading a config data. Data should be consistent at all times, even if there is invalid data in the config file. Current modules tend to store the config-related data in private structures and modify it in-place as the config file is parsed. If there is a problem, the data is left in an inconsistent state.
>
> This solution decouples config-related data from the non-config-related state held in the various private structures. It should atomically swap the global, private, and private config-related data. It also adds a higher-level API for registering the various configuration options that exist at module load, with default callback handlers for common types and the ability to create custom handlers for other types. If the higher-level API is used, a few callback functions are defined and for the most part, config loading and reloading is done with a single function call. If the high-level API is not sufficient, it can either be modified as time-goes on, or a module can use the lower-level config option API functions themselves, keeping to the same overall format of swapping out config objects, etc. for thread safe reloads.
>
> This patch also makes significant use of the RAII_VAR macro which uses the gcc "cleanup" attribute to make sure that ref counts are cleaned up on return, etc.
>
> There needs to be a lot more documentation, unit tests, etc. But I should probably hold off on doing any of that until people have had a chance to look at the basic idea, etc. There are some configs that won't work with the high-level API as-is. Anything that uses categories that have the same name (chan_mobile) would need an option added that allows that. Things like ConfBridge with options that are user-definable DTMF codes would need a "catch-all" or pattern-matching for option names. Both would be fairly easy to implement.
>
> Rationale
> --------
> Why not store config data directly in the privates?
> Because updating the data at the time of parsing can leave things in an indeterminate state on reload.
>
> What about just storing the config data directly in the privates, and creating new privates as you parse and swap out for the old one?
> Swapping out the entire private structure would lose any non-config-related state in the private structure.
>
> What about using a copy function for the private's non-config-related state?
> Having to define(and keep it updated as new fields are added) a copy function for every private structure (and essentially for every type stored in that structure) that needs to properly handle reloads sounds like a huge pain to me.
>
> What about instead of having separate containers for privates and private configs, you just store a pointer to the private config in the private structure itself?
> There are two problems I see with this. 1) To ensure data is consistent when accessing multiple fields, one would need to hold a reference to the cfg in the private. But, since it is just a pointer, it encourages people to use it directly without grabbing a reference. By separating the containers, one must look up the config object and get a reference to it to be able to use it. 2) If there is a problem in the middle of switching out the cfg pointers, you end up with some privates with new configs and some with old.
>
> Overview of how it works: You basically have the global aco_info struct that defines information pertaining to the whole config. Then there are aco_types which define category-level things like regex for what categories are supported for the type, allocation/lookup functions, whether it is for a single global object, or objects in containers, etc. Below that are aco_options, which define the options available for a given type. For example:
>
> struct aco_info cfg_info = {
> .module = AST_MODULE,
> .filename = "app_skel.conf"
> .apply_config = skel_apply_config,
> .preload = {"general", SENTINEL }, /* If you need to load some contexts in order */
> };
>
> struct skel_global_cfg {
> ...
> };
>
> struct skel_pvt_cfg {
> ...
> };
>
> struct skel_pvt {
> ...
> };
>
> enum {
> GLOBAL_OPTIONS = 0,
> PVT_CFG_CONTAINER,
> PVT_CONTAINER,
> /* Must be declared last */
> NUM_GLOBAL_OBJECTS,
> };
> static AO2_GLOBAL_OBJ_STATIC(global_config, NUM_GLOBAL_OBJECTS);
> AST_MUTEX_DEFINE_STATIC(reload_lock);
>
> /* Required for global */
> void *skel_global_cfg_alloc(const char*cat);
>
> /* Required for privates (container-stored objects) */
> void *skel_pvt_cfg_alloc(const char *cat);
> void *skel_pvt_find_or_create(const char *cat);
> void *skel_pvt_find_in_container(struct ao2_container *cont, const char *cat);
> int skel_pvt_containers_alloc(struct ao2_container **newpvts, struct ao2_container **newcfgs);
>
> /* Optional for privates */
> int skel_pvt_cfg_post_init(void *cfg); /* Could be used to inherit global settings...ew. */
> int skel_pvt_cfg_pre_link(void *cfg); /* Could be used for final verification that things look a-ok */
>
> static int apply_config(void)
> {
> RAII_VAR(void *, new_global, aco_info_new_global_get(&cfg_info, "global"), ao2_cleanup);
> RAII_VAR(struct ao2_container *, new_pvts, aco_info_new_privates_get(&cfg_info, "private"), ao2_cleanup);
> RAII_VAR(struct ao2_container *, new_cfgs, aco_info_new_configs_get(&cfg_info, "private"), ao2_cleanup);
>
> if (!(new_global && new_pvts && new_cfgs)) {
> return -1;
> }
> /* Do any fixup for global configs here, individual privates could be fixed up via the pre-link callback */
>
> ao2_global_obj_replace_unref(global_config, GLOBAL_OPTIONS, new_global);
> ao2_global_obj_replace_unref(global_config, PVT_CONTAINER, new_pvts);
> ao2_global_obj_replace_unref(global_config, PVT_CFG_CONTAINER, new_cfgs);
>
> return 0;
> }
>
> static int process_config(int reload)
> {
> ast_mutex_lock(&reload_lock);
> if (aco_process_config(&cfg_info, reload)) {...};
> ast_mutex_unlock(&reload_lock);
> ...
> }
>
> static int reload(void)
> {
> if (process_config(1)) {...}
> }
> static int load_module(void)
> {
> ...
> aco_info_init(&cfg_info));
> global_type = aco_type_global_alloc("global", CONTEXT_ALLOW, "general", (aco_type_alloc) skel_global_alloc);
> priv_type = aco_type_private_alloc("private", CONTEXT_DENY, "general", NULL, NULL, (aco_type_alloc) skel_pvt_cfg_alloc, skel_containers_alloc, skel_find_or_create_pvt, skel_find_pvt, NULL, NULL)
>
> aco_type_register(&cfg_info, global_type);
> aco_type_register(&cfg_info, priv_type);
>
> aco_option_register(&cfg_info, "foo", global_type, "booya", OPT_STRINGFIELD_T, 0, STRFLDSET(struct skel_global_config, foo));
> ...
> if (process_config(0)) {...}
> ...
> }
>
>
> Diffs
> -----
>
> /trunk/main/config_options.c PRE-CREATION
> /trunk/main/config.c 362149
> /trunk/main/asterisk.exports.in 362149
> /trunk/main/astobj2.c 362149
> /trunk/include/asterisk/utils.h 362149
> /trunk/include/asterisk/stringfields.h 362149
> /trunk/include/asterisk/config_options.h PRE-CREATION
> /trunk/include/asterisk/config.h 362149
> /trunk/include/asterisk/astobj2.h 362149
> /trunk/configs/app_skel.conf.sample PRE-CREATION
> /trunk/apps/app_skel.c 362149
> /trunk/main/udptl.c 362149
>
> Diff: https://reviewboard.asterisk.org/r/1873/diff
>
>
> Testing
> -------
>
> Lots of testing with malloc debug, valgrind, etc.
>
>
> Thanks,
>
> Terry
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20120426/4dd6bf23/attachment-0001.htm>
More information about the asterisk-dev
mailing list