[asterisk-dev] please check memory allocation sizes...

[Luigi Rizzo]

Playing with stringfields (but it is likely that astobj2 usage has
the same problem) i notice that the requested sizes of memory pools
are all powers of 2.

Unfortunately, this is the worst type of request we can make
(description below), as it tends to consume twice as much memory.
We should reduce the values to something less than 2^i (to be determined)
so that memory is used more efficiently.

The reason for the inefficiency is that allocations also include
our own overhead (for memory pools this is an extra pointer), and
on top of this the system's allocator might have its own memory
management overhead, which in turn make the block larger. Because
memory allocators typically round up requests to the next power of
2, so if you ask for a pool of size 256, once you factor in the
overhead, you are guaranteed to consume 512 bytes.

So, how much less than 2^i should we request ? It depends on the
operating system - we know the overhead for the user code (though,
as i said, asterisk code still ignores it), but not the OS overhead
(though typically is one or two pointers).

The following small program helps show the problem and find the OS
overhead for each malloc'ed block.
The first argument is the number of blocks to allocate, second is
the size of each block.

On FreeBSD, there per-block overhead is just one bit, which is allocated
elsewhere - so you can malloc 2^i blocks efficiently.
It would be great if you could check how linux variants and other OS behave,
so we can use the information to reduce the memory footprint at runtime.

thanks
luigi

----------------------------------------------------
	/* test allocation granularity */
	#include <unistd.h>
	int main(int argc, char *argv[])
	{
		const char *start = sbrk(0);
		int i, num, size;
		num = (argc > 1) ? atoi(argv[1]) : 1000;
		size = (argc > 2) ? atoi(argv[2]) : 256;
		for (i = 0; i < num; i++)
			malloc(size);
		printf("allocated %d bytes, i see %d\n",
			num*size, (const char *)sbrk(0) - start);
		return 0;
	}

------------------------------------------