[asterisk-users] puzzle

Wed Nov 19 18:19:14 CST 2008

Its not Centos - there is no 'yum'.  "service iptables stop" is what 
produced the hanging process in the first place - I think my big problem 
here is that a kernel module is broken, and there is no way to stop it, 
and there seems to be no way to unload it (in fact it is hung trying to do 
just that).

Thanks for the suggestions, though!

j

On Wed, 19 Nov 2008, Steve Totaro wrote:

> YUM update?  "service iptables stop" "service iptables start"?
>
> On Wed, Nov 19, 2008 at 4:46 PM, Jeff LaCoursiere <jeff at jeff.net> wrote:
>>
>> Hmm, I am more of a BSD guy I guess.  I would expect a pipe to show a 'p'
>> in a long ls.  This is interesting though:
>>
>> [root at ast init.d]# cat /proc/modules | head
>> ip_conntrack 45573 0 - Unloading 0xf8945000
>> [root at ast init.d]# rmmod -f ip_conntrack
>> ERROR: Removing 'ip_conntrack': Device or resource busy
>>
>> (sigh)
>>
>> I am pretty sure ip_conntrack is part of the iptables stuff...
>>
>> j
>>
>> On Wed, 19 Nov 2008, Danny Nicholas wrote:
>>
>>> /proc/modules is a pipe
>>> You can see what is in there by type cat /proc/modules|more
>>>
>>>
>>> -----Original Message-----
>>> From: asterisk-users-bounces at lists.digium.com
>>> [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Jeff
>>> LaCoursiere
>>> Sent: Wednesday, November 19, 2008 2:47 PM
>>> To: Asterisk Users Mailing List - Non-Commercial Discussion
>>> Subject: Re: [asterisk-users] puzzle
>>>
>>>
>>> A good idea!  The modprobe command is actually in the ps below - it is
>>> part of the /etc/init.d/iptables script, and apparently was trying to
>>> remove the ipt_state module.  The result, however:
>>>
>>> [root at ast init.d]# rmmod ipt_state
>>> ERROR: Module ipt_state does not exist in /proc/modules
>>>
>>> (sigh).  In fact /proc/modules is empty.
>>>
>>> [root at ast init.d]# ls -ltr /proc/modules
>>> -r--r--r--  1 root root 0 Nov 19 14:46 /proc/modules
>>>
>>> j
>>>
>>> On Wed, 19 Nov 2008, Danny Nicholas wrote:
>>>
>>>> Your could try this
>>>> History|grep modprobe
>>>> Rmmod XXX where xxx is the parameter from the history|grep modprobe.
>>>> This of course assumes that the command is in your last 1000 commands.
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: asterisk-users-bounces at lists.digium.com
>>>> [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Jeff
>>>> LaCoursiere
>>>> Sent: Wednesday, November 19, 2008 2:20 PM
>>>> To: Asterisk Users Mailing List - Non-Commercial Discussion
>>>> Subject: Re: [asterisk-users] puzzle
>>>>
>>>>
>>>> Yes, the second 'ps' below showed the parent to be '1' (init), which means
>>>> its real parent died already.
>>>>
>>>> Any attempt to flush the iptables hangs :(
>>>>
>>>> j
>>>>
>>>> On Wed, 19 Nov 2008, Danny Nicholas wrote:
>>>>
>>>>> Have you done a ps -elf to see if the process has a parent that is
>>>>> re-launching or preserving it?
>>>>>
>>>>> -----Original Message-----
>>>>> From: asterisk-users-bounces at lists.digium.com
>>>>> [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Jeff
>>>>> LaCoursiere
>>>>> Sent: Wednesday, November 19, 2008 1:58 PM
>>>>> To: asterisk-users at lists.digium.com
>>>>> Subject: [asterisk-users] puzzle
>>>>>
>>>>>
>>>>> Sorry again for the only marginal relation to asterisk, but the issue
>>> does
>>>>> affect the voice performance I am experiencing, so I am soothing my guilt
>>>>> with that.
>>>>>
>>>>> Bet you don't see this every day:
>>>>>
>>>>> ast% uptime
>>>>>  13:48:08 up 981 days, 18:29,  1 user,  load average: 1.08, 1.02, 1.01
>>>>> ast%
>>>>>
>>>>> I *REALLY* want this machine to see 1000 days uptime, if for nothing
>>> other
>>>>> than bragging rights.  Its been through mysql and asterisk upgrades, a
>>>>> horrible hacking nightmare that very nearly made me reboot, and several
>>>>> power outages where the batteries lasted JUST long enough to keep her up.
>>>>>
>>>>> After all of this, I find I may have to reboot after all.  Because there
>>>>> is a !$@#% process running, consuming 100% CPU (note the load average),
>>>>> and I cannot seem to kill it:
>>>>>
>>>>> ast% ps auxw | grep modprobe
>>>>> root     17744 99.9  0.0  2688  412 ?        RN   Nov03 23223:01 modprobe
>>>>> -r ipt_state
>>>>> ast% ps ealx | grep modprobe | grep -v grep
>>>>> 4     0 17744     1  39  19  2688  412 -      RN   ?        23223:38
>>>>> modprobe -r ipt_state
>>>>> ast% sudo kill 17744
>>>>> ast% sudo kill 17744
>>>>> ast% sudo kill -9 17744
>>>>> ast% sudo kill -9 17744
>>>>> ast% !ps
>>>>> ps ealx | grep modprobe | grep -v grep
>>>>> 4     0 17744     1  39  19  2688  412 -      RN   ?        23224:41
>>>>> modprobe -r ipt_state
>>>>> ast%
>>>>>
>>>>> You may also notice that I tried "renice" to bump it all the way to +19
>>>>> and still it consumes 100% of the CPU.  The result for asterisk is that I
>>>>> hear bits of robot noise during conversations, which is annoying as hell
>>>>> but not neccessarily show stopping.  But for another 19 days??  Argg!
>>>>>
>>>>> I assume that because it is 'modprobe' it has tickled some kernel bug
>>> that
>>>>> is merrily spinning away and won't respond to interrupts.  I even tried
>>> to
>>>>> stop it with gdb and strace, both of which also hung and had to be killed
>>>>> with -9.
>>>>>
>>>>> It seems to be related to me screwing with the iptables a few weeks ago.
>>>>>
>>>>> Any ideas other than rebooting?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> j
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>>>
>>>>> asterisk-users mailing list
>>>>> To UNSUBSCRIBE or update options visit:
>>>>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>>>
>>>>> asterisk-users mailing list
>>>>> To UNSUBSCRIBE or update options visit:
>>>>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>>>>
>>>>
>>>> _______________________________________________
>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>>
>>>> asterisk-users mailing list
>>>> To UNSUBSCRIBE or update options visit:
>>>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>>>
>>>>
>>>> _______________________________________________
>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>>
>>>> asterisk-users mailing list
>>>> To UNSUBSCRIBE or update options visit:
>>>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>>>
>>>
>>> _______________________________________________
>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>
>>> asterisk-users mailing list
>>> To UNSUBSCRIBE or update options visit:
>>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>>
>>>
>>> _______________________________________________
>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>
>>> asterisk-users mailing list
>>> To UNSUBSCRIBE or update options visit:
>>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>>
>>
>> _______________________________________________
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> asterisk-users mailing list
>> To UNSUBSCRIBE or update options visit:
>>   http://lists.digium.com/mailman/listinfo/asterisk-users
>>
>
>
>
> -- 
> Thanks,
> Steve Totaro
> +18887771888 (Toll Free)
> +12409381212 (Cell)
> +12024369784 (Skype)
>
> _______________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>