[asterisk-dev] [Code Review]: ensure that ast_string_field_pool base + used is always aligned

Tue Nov 1 10:35:16 CDT 2011

> On Oct. 31, 2011, 12:34 p.m., mjordan wrote:
> > /branches/1.8/main/utils.c, line 1548
> > <https://reviewboard.asterisk.org/r/1549/diff/2/?file=21505#file21505line1548>
> >
> >     This actually goes against the recommendations in the coding guidelines (2.17.3) - is there a specific reason to not use ast_calloc?
> 
> David Vossel wrote:
>     The memset after the malloc is for a smaller amount of memory than the entire allocation.  The change appears safe, but I question why we'd even care about this optimization when it adds more complexity.
> 
> wdoekes wrote:
>     This isn't an increase in complexity, this is a fix. The original author should never have used calloc here. This is a very common pattern.
>     
>     I don't see how you could not care about the overhead of hundreds of bytes getting nulled for nothing when only a handful (struct ast_string_field_pool) should get nulled. Especially in core functions that were created to reduce load (individual string field mallocs) in the first place.
> 
> Terry Wilson wrote:
>     From the coding guidelines:
>     When allocating/zeroing memory for a structure, use code like this:
>     
>     
>     struct foo *tmp;
>     
>     ...
>     
>     tmp = ast_calloc(1, sizeof(*tmp));
>     Avoid the combination of ast_malloc() and memset(). Instead, always use ast_calloc(). This will allocate and zero the memory in a single operation. In the case that uninitialized memory is acceptable, there should be a comment in the code that states why this is the case.
>     
>     Considering that this would be initialization code and not something that would be run in a tight loop, the optimization isn't a particularly big deal. We decided long ago to err on the side of caution to protect against people checking if (!foo->bar) against unitialized data. In this case, I would assume that it would be safe since __ast_string_field_init initializes the strings to empty (if I'm reading it correctly, I haven't looked at the stringfields code much). We just have a strong preference to using calloc as a general rule unless it is in an area where performance is really greatly affected during normal runtime.
> 
> Terry Wilson wrote:
>     Re-reading that it sounds like I might be against the optimization when I'm not. I was mostly explaining why we normally prefer calloc. If we are indeed initializing with calloc and re-initializing in the init function, that seems kind of silly. We should add a comment explaining that the malloc/memset is done on purpose as it deviates from our standard allocation procedures.
> 
> Tilghman Lesher wrote:
>     I'm not against the replacement of code, either, but Walter has a misunderstanding of why string fields were created.  It was not intended to reduce load, but to reduce the amount of memory that these core structures took.  Originally, these structures were created as a series of static buffers, the result of which was that (for example) channel names were both limited in size for the more extreme users, as well as mostly unused for most users.  The compromise was to create stringfields, which compacted the amount of memory used, still allowing longer strings for extreme users, while also ensuring that the number of mallocs did not significantly increase.
> 
> David Vossel wrote:
>     In general, unless we can actually quantify the performance gained by moving a calloc to a malloc and using memset I am personally against the change.  Here is my reasoning.  If the calloc has been used since the beginning of time, we always run a risk that somewhere someone is depending on the memory being initialized simply because it always has been, regardless if it should have been or not.
>     
>     In this case it looks safe, but I am not familiar enough with stringfields to say it is 100% safe without auditing all the stringfield code.  Maybe someone else is that familiar with the code and will give it a ship it.  Regardless it runs the risk of regression with practically no benefit.
>     
>     Don't take this to mean I am against code optimization.  If this code was re-initializing hundreds of bytes used as the mixing buffer for a conference every 20 ms, this would run a similar risk of regression but the performance benefit would be worthwhile in my opinion.
>     
>     If you go auditing the code for improvements, you will find them.  I never want to discourage that.  If someone knows the stringfield code well enough to give this a ship it, then go for it.  I just want you to understand the risk of changing things as simple as memory initialization in a codebase as complex as ours.

Looking at my first statement in the previous post, moving a calloc to a malloc with memset will not cause a regression.  I was just referring to the change at hand were we change the calloc to use malloc and used memset on a subset of the allocated space for anyone who didn't look at the code in question.

- David

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/1549/#review4615
-----------------------------------------------------------

On Oct. 31, 2011, 3:47 a.m., wdoekes wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/1549/
> -----------------------------------------------------------
> 
> (Updated Oct. 31, 2011, 3:47 a.m.)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Summary
> -------
> 
> This patch fixes that Asterisk can be properly built on certain architectures that dislike misalignment. (In the case of the bug reporter, an ARM.)
> 
> ==Background==
> Currently the 16bit ast_string_field_allocation used in the is not aligned, it can be stored on an 8bit boundary. Certain machines will either SIGBUS over this or simply give wrong results. For the Sparc an #ifdef was added to alleviate the problem.
> 
> ==Problems with current approach==
> (1) The x86 can cope with misaligned integers, but for performance, aligned ints are better.
> (2) The #ifdef did not catch all architectures that dislike misalignment.
> (3) The code in the #ifdef falsely assumes that the ast_string_field_allocation is at most 2 bytes large. If this were to change one day, things would start to fail again.
> 
> ==Possible fixes==
> (1) Remove the #ifdef, always run the Sparc code and patch it to cope with larger than 16bit ast_string_field_allocation's.
> (2) Alter all ast_string_field_allocation code to ensure that base and used stay aligned. Then we won't need to check and re-align later on.
> 
> I chose fix #2 because I believe this to be marginally faster and more logical. This does involve the use of the gcc __attribute__((aligned)). But the other code is full of gcc attributes, so I don't think I'm breaking a build anywhere with this.
> 
> Regards,
> Walter
> 
> 
> This addresses bug ASTERISK-17310.
>     https://issues.asterisk.org/jira/browse/ASTERISK-17310
> 
> 
> Diffs
> -----
> 
>   /branches/1.8/include/asterisk/utils.h 342659 
>   /branches/1.8/main/utils.c 342659 
>   /branches/1.8/include/asterisk/stringfields.h 342659 
> 
> Diff: https://reviewboard.asterisk.org/r/1549/diff
> 
> 
> Testing
> -------
> 
> I replaced:
> typedef uint16_t ast_string_field_allocation;
> with:
> typedef uint64_t ast_string_field_allocation;
> 
> Then I looked at a small sample of base and used during operation.
> 
> They were always 64bit aligned.
> 
> 
> Thanks,
> 
> wdoekes
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20111101/7aa8a535/attachment-0001.htm>