[asterisk-dev] asterisk 1.4 rtp port exhaustion fix

[Tilghman Lesher]

On Tuesday 17 May 2011 12:08:46 Paul Albrecht wrote:
> On Tue, 2011-05-17 at 11:43 -0500, Tilghman Lesher wrote:
> > On Tuesday 17 May 2011 09:10:34 Paul Albrecht wrote:
> > > On Mon, 2011-05-16 at 16:42 -0500, Tilghman Lesher wrote:
> > > > On Monday 16 May 2011 15:26:29 Paul Albrecht wrote:
> > > > > On Mon, 2011-05-16 at 13:53 -0500, Paul Belanger wrote:
> > > > > > On 11-05-16 02:02 PM, Paul Albrecht wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > When I run the mp3 application I sometimes run out of rtp
> > > > > > > ports because asterisk doesn't close open files and sockets
> > > > > > > before exec'ing the mpg123
> > > > > > 
> > > > > > > process. Here's a fix:
> > > > > > If there is not already an issue on the tracker, please create
> > > > > > one. Also, uploading your patch to reviewboard would be
> > > > > > recommended.
> > > > > 
> > > > > There must be an issue already because someone "fixed" the
> > > > > problem for later asterisk releases. The thing is I don't think
> > > > > the "fix" makes sense for linux which I thought was the
> > > > > preferred platform for asterisk. The linuxy way to close open
> > > > > descriptors before exec'ing a script would be to set the "close
> > > > > on exec" whenever a file is opened or a socket is created. As
> > > > > for the review board, I simply fixed the stuff that affected
> > > > > the applications that I support so the codes not finished and
> > > > > not ready for review.
> > > > 
> > > > We opted in the later releases to make it part of the API and
> > > > standardize the various methods to ensure that all file
> > > > descriptors were closed (as well as fixing signal races and other
> > > > issues).  No, it's not Linux-specific, but it should work fine on
> > > > all platforms where fork is available (as opposed to vfork).
> > > > 
> > > > As to the CLOEXEC method, it's very difficult to make that method
> > > > work reliably, because many calls that open file descriptors do
> > > > not have flags to ensure that the CLOEXEC flag is set atomically.
> > > > Additionally, linked libraries may open file descriptors at
> > > > various intervals, and it would be impossible to ensure that each
> > > > of those libraries also used the CLOEXEC flag.  It was definitely
> > > > considered at the time, but it was rejected, because it's not a
> > > > reliable method.
> > > 
> > > Don't know what you mean when you say the "close on exec" flag is
> > > unreliable. Can you point out which code sequences you think are
> > > problematic in the patch I submitted to this list?
> > 
> > No, I cannot.  We do not look at patches submitted to the list, only
> > patches submitted with a license on the issue tracker.  However, I've
> > outlined above the reasons why it's not reliable to set the flag on
> > the file descriptor when the descriptor is opened.  There's also an
> > inherent race between two forked subprocesses, each of which will
> > create file descriptors without the CLOEXEC flag (because the file
> > descriptors need to remain open after exec).
> 
> Nonsense. You made an assertion of fact which you can not or will not
> support with evidence and that is simply ridiculous. If you can't
> comment of the code, why did you respond to my post in the first place?

I replied because I wrote the organized fork code in Asterisk.  I've
already specified the reasons for not using CLOEXEC, above.  I'll be a
little more explicit, then.  The problem is that as Asterisk is a
multi-threaded application, many file descriptors are being opened and
closed simultaneously.  A good number of the system calls that open file
descriptors don't support setting the CLOEXEC flag in the same call (or
don't in a portable fashion, including Solaris, FreeBSD, Mac OS X, and
others).  That's important, because if a fork(2) occurs after the file
descriptor is opened but before the CLOEXEC flag can be set, the file
descriptor will remain open in the child after the exec.  That's why
instead of trying to manage the flags, it's just easier to close them in the
child.  The only real issue with this technique is determining the maximum
file descriptor to close. 

> Moreover, your assertion that the "close on exec" flag doesn't cover the
> case where a library that asterisk calls creates a file or socket hardly
> supports your argument. Why should asterisk close open file or sockets
> it doesn't own?

Because after exec, the library-opened descriptors aren't germane to the
process, but they will remain open if not explicitly closed.  Since the
issue is a leak of file descriptors, we have to ensure that we take care of
all of them, even if we don't explicitly know about them.

-- 
Tilghman