[asterisk-users] Re: Kernel Panic in wct4xxp during unload on
Zaptel-1.4.4
Axel Thimm
Axel.Thimm at ATrpms.net
Tue May 22 08:16:11 MST 2007
On Tue, May 22, 2007 at 10:07:19AM -0400, James FitzGibbon wrote:
> I attempted an upgrade of our production system from Asterisk/Zaptel 1.2 to
> 1.4 this weekend. Intially everything looked like it was working properly,
> but some time in the day following the upgrade, the system died to a kernel
> panic. I wasn't able to catch the entire kernel dump on the console
> unfortunately.
>
> I attempted to isolate the panic, and found that when 'service zaptel stop'
> was run (specifically, when wct4xxp was unloaded) I get this panic
> consistently:
>
> <5> Not prepped yet! (repeated approx 250 times)
> <5> Freed a Wildcard
> <5> Not prepped yet! (repeated approx 550 times)
> <5> Stopped TE4XXP, Turned off DMA
> <5> Not prepped yet! (repeated approx 11000 times)
> <5> Unable to handle kernel paging request at ffffff0000034010 RIP:
> <5> <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63}
> <5> PML4 4ea063 PGD 1388067 PMD 1389067 PTE 0
> <5> Oops: 0000 [1] SMP
> <5> CPU 1
> <5> Modules linked in: zttranscode(U) wct4xxp(U) zaptel(U) crc_ccitt
> netconsole md5 ipv6 dm_mirror dm_mod button battery ac joydev ehci_hcd
> uhci_hcd hw_random bnx2 ext3 jbd
> cciss sd_mod scsi_mod
> <5> Pid: 11053, comm: hotplug Not tainted 2.6.9-42.0.8.ELsmp
> <5> RIP: 0010:[<ffffffffa0163207>]
> <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63}
> <5> RSP: 0000:00000100013ebdb0 EFLAGS: 00010046
> <5> RAX: ffffff0000034000 RBX: 0000010073478724 RCX: 0000000000000002
> <5> RDX: 0000010073478680 RSI: 0000000000000002 RDI: 0000010073478724
> <5> RBP: 0000010073478680 R08: 0000000000000008 R09: 0000000000000000
> <5> R10: 0000000000000000 R11: 0000000000000002 R12: 00000000000000d1
> <5> R13: 00000100013ebec8 R14: 00000100013ebec8 R15: 0000010075e94978
> <5> FS: 0000002a955643e0(0000) GS:ffffffff804e5900(0000)
> knlGS:0000000000000000
> <5> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> <5> CR2: ffffff0000034010 CR3: 00000000013d8000 CR4: 00000000000006e0
> <5> Process hotplug (pid: 11053, threadinfo 000001007413c000, task
> 000001007c445030)
> <5> Stack: 00000000000000d1 000001007413dc98 ffffffff80138552
> 0000003000000008
> <5> 00000100013ebea8 00000100013ebde8 0000000000000001
> 00000000000000d1
> <5> 0000000000000012 0000010073478680
> <5> Call Trace:<IRQ> <ffffffff80138552>{printk+141}
> <ffffffff80112f4a>{handle_IRQ_event+41}
> <5> <ffffffff801131c4>{do_IRQ+197}
> <ffffffff80110833>{ret_from_intr+0}
> <5> <ffffffff8013c731>{__do_softirq+77}
> <ffffffff8013c7e5>{do_softirq+49}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133} <EOI>
> <ffffffff8011c21a>{flush_tlb_page+44}
> <5> <ffffffff80169106>{do_wp_page+1127}
> <ffffffff80123ed3>{do_page_fault+575}
> <5> <ffffffff80169ff2>{handle_mm_fault+1228}
> <ffffffff80123e9a>{do_page_fault+518}
> <5> <ffffffff8011026a>{system_call+126}
> <ffffffff80132bc6>{schedule_tail+202}
> <5> <ffffffff80110d91>{error_exit+0}
> <5>
> <5> Code: 8b 40 10 89 44 24 58 e8 3d 80 1a e0 31 c0 f6 44 24 58 07 0f
> <5> RIP <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63} RSP
> <00000100013ebdb0>
> <5> CR2: ffffff0000034010
> <5> <0>Kernel panic - not syncing: Oops
> <5> Badness in panic at kernel/panic.c:118
> <5>
> <5> Call Trace:<IRQ> <ffffffff80137a8a>{panic+527}
> <ffffffff80110bf5>{apic_timer_interrupt+133}
> <5> <ffffffff80111aec>{oops_end+38} <ffffffff80111b07>{oops_end+65}
> <5> <ffffffff80124148>{do_page_fault+1204}
> <ffffffffa0078f51>{:bnx2:bnx2_start_xmit+470}
> <5> <ffffffff802bb4cd>{netpoll_send_skb+257}
> <ffffffff80110d91>{error_exit+0}
> <5> <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63}
> <ffffffff80138552>{printk+141}
> <5> <ffffffff80112f4a>{handle_IRQ_event+41}
> <ffffffff801131c4>{do_IRQ+197}
> <5> <ffffffff80110833>{ret_from_intr+0}
> <ffffffff8013c731>{__do_softirq+77}
> <5> <ffffffff8013c7e5>{do_softirq+49}
> <ffffffff80110bf5>{apic_timer_interrupt+133}
> <5> <EOI> <ffffffff8011c21a>{flush_tlb_page+44}
> <ffffffff80169106>{do_wp_page+1127}
> <5> <ffffffff80123ed3>{do_page_fault+575}
> <ffffffff80169ff2>{handle_mm_fault+1228}
> <5> <ffffffff80123e9a>{do_page_fault+518}
> <ffffffff8011026a>{system_call+126}
> <5> <ffffffff80132bc6>{schedule_tail+202}
> <ffffffff80110d91>{error_exit+0}
> <5>
> <5> Badness in i8042_panic_blink at drivers/input/serio/i8042.c:987
> <5>
> <5> Call Trace:<IRQ> <ffffffff8024219b>{i8042_panic_blink+238}
> <ffffffff80137a38>{panic+445}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133}
> <ffffffff80111aec>{oops_end+38}
> <5> <ffffffff80111b07>{oops_end+65}
> <ffffffff80124148>{do_page_fault+1204}
> <5> <ffffffffa0078f51>{:bnx2:bnx2_start_xmit+470}
> <ffffffff802bb4cd>{netpoll_send_skb+257}
> <5> <ffffffff80110d91>{error_exit+0}
> <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63}
> <5> <ffffffff80138552>{printk+141}
> <ffffffff80112f4a>{handle_IRQ_event+41}
> <5> <ffffffff801131c4>{do_IRQ+197}
> <ffffffff80110833>{ret_from_intr+0}
> <5> <ffffffff8013c731>{__do_softirq+77}
> <ffffffff8013c7e5>{do_softirq+49}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133} <EOI>
> <ffffffff8011c21a>{flush_tlb_page+44}
> <5> <ffffffff80169106>{do_wp_page+1127}
> <ffffffff80123ed3>{do_page_fault+575}
> <5> <ffffffff80169ff2>{handle_mm_fault+1228}
> <ffffffff80123e9a>{do_page_fault+518}
> <5> <ffffffff8011026a>{system_call+126}
> <ffffffff80132bc6>{schedule_tail+202}
> <5> <ffffffff80110d91>{error_exit+0}
> <5> Badness in i8042_panic_blink at drivers/input/serio/i8042.c:990
> <5>
> <5> Call Trace:<IRQ> <ffffffff8024222d>{i8042_panic_blink+384}
> <ffffffff80137a38>{panic+445}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133}
> <ffffffff80111aec>{oops_end+38}
> <5> <ffffffff80111b07>{oops_end+65}
> <ffffffff80124148>{do_page_fault+1204}
> <5> <ffffffffa0078f51>{:bnx2:bnx2_start_xmit+470}
> <ffffffff802bb4cd>{netpoll_send_skb+257}
> <5> <ffffffff80110d91>{error_exit+0}
> <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63}
> <5> <ffffffff80138552>{printk+141}
> <ffffffff80112f4a>{handle_IRQ_event+41}
> <5> <ffffffff801131c4>{do_IRQ+197}
> <ffffffff80110833>{ret_from_intr+0}
> <5> <ffffffff8013c731>{__do_softirq+77}
> <ffffffff8013c7e5>{do_softirq+49}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133} <EOI>
> <ffffffff8011c21a>{flush_tlb_page+44}
> <5> <ffffffff80169106>{do_wp_page+1127}
> <ffffffff80123ed3>{do_page_fault+575}
> <5> <ffffffff80169ff2>{handle_mm_fault+1228}
> <ffffffff80123e9a>{do_page_fault+518}
> <5> <ffffffff8011026a>{system_call+126}
> <ffffffff80132bc6>{schedule_tail+202}
> <5> <ffffffff80110d91>{error_exit+0}
> <5> Badness in i8042_panic_blink at drivers/input/serio/i8042.c:992
> <5>
> <5> Call Trace:<IRQ> <ffffffff80242292>{i8042_panic_blink+485}
> <ffffffff80137a38>{panic+445}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133}
> <ffffffff80111aec>{oops_end+38}
> <5> <ffffffff80111b07>{oops_end+65}
> <ffffffff80124148>{do_page_fault+1204}
> <5> <ffffffffa0078f51>{:bnx2:bnx2_start_xmit+470}
> <ffffffff802bb4cd>{netpoll_send_skb+257}
> <5> <ffffffff80110d91>{error_exit+0}
> <ffffffffa0163207>{:wct4xxp:t4_interrupt_gen2+63}
> <5> <ffffffff80138552>{printk+141}
> <ffffffff80112f4a>{handle_IRQ_event+41}
> <5> <ffffffff801131c4>{do_IRQ+197}
> <ffffffff80110833>{ret_from_intr+0}
> <5> <ffffffff8013c731>{__do_softirq+77}
> <ffffffff8013c7e5>{do_softirq+49}
> <5> <ffffffff80110bf5>{apic_timer_interrupt+133} <EOI>
> <ffffffff8011c21a>{flush_tlb_page+44}
> <5> <ffffffff80169106>{do_wp_page+1127}
> <ffffffff80123ed3>{do_page_fault+575}
> <5> <ffffffff80169ff2>{handle_mm_fault+1228}
> <ffffffff80123e9a>{do_page_fault+518}
> <5> <ffffffff8011026a>{system_call+126}
> <ffffffff80132bc6>{schedule_tail+202}
> <5> <ffffffff80110d91>{error_exit+0}
>
> System details:
>
> HP DL380G5
> CentOS x86_64 4.4
> uname: 2.6.9-42.0.8.ELsmp #1 SMP Tue Jan 30 12:18:01 EST 2007 x86_64 x86_64
> x86_64 GNU/Linux
> T412P quad-span card
> TDM400 card with two FXS modules and two FXO modules
>
> Original software:
> Libpri 1.2.4 (RPM from atrpms)
> Zaptel 1.2.16 (RPM from atrpms)
> Asterisk 1.2.17 (compiled from source)
>
> Upgraded software:
> Libpri 1.4.0 (compiled from source)
> Zaptel 1.4.2.1 (compiled from source)
> Asterisk 1.4.4 (compiled from source)
Have you tried using the 1.4.x atrpms packages?
> Before installing the 1.4 files, I removed the 1.2 RPMs using 'yum remove',
> so there should not have been any cruft left over. I did notice that the
> atrpms tarballs put some files in different locations. The RPM for libpri
> puts the files in /usr/lib64, while my source tarball put them in /usr/lib.
> 'file' indicates that they are both 'ELF 64-bit LSB shared object, AMD
> x86-64, version 1 (SYSV)' though, so I don't think that this is an issue of
> mixed 32 and 64 bit objects.
>
> I tried rebuilding all of the software from cleanly extracted tarballs to no
> avail, so I had to restore my backed up 1.2 configuration. Things still
> aren't working properly: when I attempt to unload the wct4xxp module, the
> "Not prepped yet!" message floods the console and the 'rmmod' command
> becomes hung (unkillable), but the system does not panic. I still have to
> hard reboot to get the system in a state where I can bring up * again
> though.
>
> Since the panic seems to involve the T412P interrupts, here's the output of
> /proc/interrupts:
>
> [root at pbxtel-01 ~]# cat /proc/interrupts
> CPU0 CPU1
> 0: 799008 804849 IO-APIC-edge timer
> 1: 4 5 IO-APIC-edge i8042
> 8: 5 1 IO-APIC-edge rtc
> 9: 0 0 IO-APIC-level acpi
> 74: 16224 3990 PCI-MSI-X cciss0
> 90: 154441 0 PCI-MSI eth0
> 169: 0 0 IO-APIC-level uhci_hcd, ehci_hcd
> 177: 0 0 IO-APIC-level uhci_hcd
> 185: 0 0 IO-APIC-level uhci_hcd
> 193: 0 0 IO-APIC-level uhci_hcd
> 201: 814161 720175 IO-APIC-level wctdm
> 209: 720174 814169 IO-APIC-level wct4xxp
> 233: 34 47 IO-APIC-level uhci_hcd
> NMI: 1603725 1603680
> LOC: 1603015 1603015
> ERR: 0
> MIS: 0
> [root at pbxtel-01 ~]#
>
> (this is with the system as it is running now with the backed out to
> 1.2configuration)
>
> Any thoughts?
>
> Thanks
--
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.digium.com/pipermail/asterisk-users/attachments/20070522/3aa868d8/attachment.pgp
More information about the asterisk-users
mailing list