[asterisk-dev] policy change for correct handling of fatal program conditions

[Paul Hewlett]

On Tuesday 23 January 2007 14:15, SF Markus Elfring wrote:
> >    Has anyone considered the fact that malloc() never actually fails i.e.
> > memory starvation does not cause malloc() to return error. It always
> > returns true - when you actually access the memory one will get a SIGBUS
> > (I think) and the program will die.
> >
> >   This behaviour (called overcommit) is to make the UNIX fork/exec model
> > efficient. For this reason one should use calloc() because at least then
> > the failure will occur close to the allocation call in the code.
>
> Do you refer to the Linux and FreeBSD implementation?
Hi Marcus

Yes - although other UNIX's have had this feature in the past.

I was in error in my comments above - when the kernel detects memory 
starvation, the dreaded OOM killer is called which attempts to kill processes 
that are memory hogs. The original OOM killer without fail would kill the X 
server if it was running. Later versions tried to be cleverer than this.

There was kernel patch that allowed one to exempt ceratin processes from the 
OOM killer - I don't think it made it into the main tree however. asterisk 
would be a candidate for this.

> http://baus.net/memory-management
> http://www.ussg.iu.edu/hypermail/linux/kernel/0311.3/0420.html
> http://homepage.mac.com/mguthaus/tips/2005/tip050903.html
>
> >   There have been many discussions about this quirk of malloc. The man
> > page even describes it as a bug ...
>
> To which manual do you refer to?

Type 'man malloc' on the linux box....

See also 
http://developers.sun.com/solaris/articles/subprocess/subprocess.html#overcom

There are some references at the end - the RedHat article incorrectly 
describes option 0 - the correct description is 
in /usr/src/linux/Documentation/vm/overcommit-accounting


> Is it a similar version like this one?
> http://sman.informatik.htw-dresden.de:6711/man?=malloc
can't get that page
>

Linus has ranted about this strange quirk but eventually allowed the extra 
sysctl settings.
(http://mail.python.org/pipermail/python-dev/1995-March/008945.html)

Basically the problem is that the sbrk() call (on which malloc() is based) 
which gets memory pages from the kernel never returns them to the kernel. So 
once allocated to a process the only way to get the memory pages  back into 
the kernel pool is to kill the process that allocated it.

So if a memory hog process wants to spawn a low-memory process, it can do so 
without needing to duplicate all that memory (so-called copy-on-write)

This means that free() only returns memory to a pool of bytes that are only 
available to the process for subsequent mallocs and not to any other 
processes. 

Some people write code that does all the allocations needed at startup. If 
there is not enough resources available, then the program will fail 
immediately instead of at some arbitrary later time - this minimises the 
surprise factor.

Paul
-- 
Paul Hewlett  Technical Director 
Global Call Center Solutions Ltd, 2nd Floor, Milnerton Mall
Cnr Loxton & Koeberg Roads, 7435 Milnerton
paul at gccs.co.za  www.gccs.co.za
Tel: +27 86 111 3433 Fax: +27 86 111 3520 Cel: +27 76 072 7906
Gizmo: 1 747 659 6171