<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hello all-
<div><br>
</div>
<div>I have a Digium TE410P in an HP DL145 G2 dual processor server that generates well over 100,000 interrupts per second (sometimes I’ve counted 160,000+ per second) generally resulting in either the system becoming swamped and unusable or the kernel disabling
the IRQ the TE410P is on resulting in the spans on that card being unusable.</div>
<div><br>
</div>
<div>I have confirmed that the card is good by placing it in an IBM server running FreePBX Distro and verifying that it generates only 1,000 interrupts per second, and works properly.</div>
<div><br>
</div>
<div>This is on a system running 64-bit Ubuntu 14.04 LTS, kernels 3.13.0-27-generic and 3.13.0-27-lowlatency. I have compiled and installed DAHDI from source, both 2.9.1.1 and 2.8.0, and see the same result with the Ubuntu DAHDI package which is based on 2.5.0.
I have entered BIOS and disabled all extra devices I can and reset the configuration data.</div>
<div><br>
</div>
<div>Most frequently the interrupt is disabled by the kernel - booting with the irqpoll option as suggested by the error message does not always solve the problem and introduces other problems. See dmesg below:</div>
<div><br>
</div>
<div><font face="Courier New">(not prepped yet message repeat *many* times)</font></div>
<div>
<div><font face="Courier New">[ 16.371739] wct4xxp 0000:81:01.0: Not prepped yet!</font></div>
<div><font face="Courier New">[ 16.371743] wct4xxp 0000:81:01.0: Not prepped yet!</font></div>
<div><font face="Courier New">[ 16.611991] irq 25: nobody cared (try booting with the "irqpoll" option)</font></div>
<div><font face="Courier New">[ 16.615221] CPU: 0 PID: 0 Comm: swapper/0 Tainted: GF O 3.13.0-27-generic #50-Ubuntu</font></div>
<div><font face="Courier New">[ 16.615224] Hardware name: HP ProLiant DL145 G2/K85NL, BIOS 2.14 10/20/2005</font></div>
<div><font face="Courier New">[ 16.615227] ffff880139ea6a9c ffff88013bc03e68 ffffffff817199c4 ffff880139ea6a00</font></div>
<div><font face="Courier New">[ 16.615231] ffff88013bc03e90 ffffffff810c19d2 ffff880139ea6a00 0000000000000019</font></div>
<div><font face="Courier New">[ 16.615235] 0000000000000000 ffff88013bc03ed0 ffffffff810c1e6c 000000008101b763</font></div>
<div><font face="Courier New">[ 16.615239] Call Trace:</font></div>
<div><font face="Courier New">[ 16.615241] <IRQ> [<ffffffff817199c4>] dump_stack+0x45/0x56</font></div>
<div><font face="Courier New">[ 16.615253] [<ffffffff810c19d2>] __report_bad_irq+0x32/0xd0</font></div>
<div><font face="Courier New">[ 16.615257] [<ffffffff810c1e6c>] note_interrupt+0x1ac/0x200</font></div>
<div><font face="Courier New">[ 16.615260] [<ffffffff810bf749>] handle_irq_event_percpu+0xd9/0x1d0</font></div>
<div><font face="Courier New">[ 16.615263] [<ffffffff810bf87d>] handle_irq_event+0x3d/0x60</font></div>
<div><font face="Courier New">[ 16.615267] [<ffffffff810c29ea>] handle_fasteoi_irq+0x5a/0x100</font></div>
<div><font face="Courier New">[ 16.615272] [<ffffffff81015cde>] handle_irq+0x1e/0x30</font></div>
<div><font face="Courier New">[ 16.615276] [<ffffffff8172c6cd>] do_IRQ+0x4d/0xc0</font></div>
<div><font face="Courier New">[ 16.615281] [<ffffffff81721e6d>] common_interrupt+0x6d/0x6d</font></div>
<div><font face="Courier New">[ 16.615283] <EOI> [<ffffffff810d63c1>] ? tick_nohz_idle_enter+0x41/0x70</font></div>
<div><font face="Courier New">[ 16.615289] [<ffffffff810d63bd>] ? tick_nohz_idle_enter+0x3d/0x70</font></div>
<div><font face="Courier New">[ 16.615292] [<ffffffff810beb48>] cpu_startup_entry+0x88/0x290</font></div>
<div><font face="Courier New">[ 16.615297] [<ffffffff81707e97>] rest_init+0x77/0x80</font></div>
<div><font face="Courier New">[ 16.615302] [<ffffffff81d35f70>] start_kernel+0x438/0x443</font></div>
<div><font face="Courier New">[ 16.615305] [<ffffffff81d35941>] ? repair_env_string+0x5c/0x5c</font></div>
<div><font face="Courier New">[ 16.615308] [<ffffffff81d35120>] ? early_idt_handlers+0x120/0x120</font></div>
<div><font face="Courier New">[ 16.615312] [<ffffffff81d355ee>] x86_64_start_reservations+0x2a/0x2c</font></div>
<div><font face="Courier New">[ 16.615315] [<ffffffff81d35733>] x86_64_start_kernel+0x143/0x152</font></div>
<div><font face="Courier New">[ 16.615317] handlers:</font></div>
<div><font face="Courier New">[ 16.615987] [<ffffffffa01d3420>] t4_interrupt_gen2 [wct4xxp]</font></div>
<div><font face="Courier New">[ 16.615987] Disabling IRQ #25</font></div>
<div><font face="Courier New">[ 17.607238] dahdi_echocan_mg2: Registered echo canceler 'MG2'</font></div>
<div><font face="Courier New">[ 17.608276] wct4xxp 0000:81:01.0: Span 1 configured for ESF/B8ZS</font></div>
<div><font face="Courier New">[ 17.608360] wct4xxp 0000:81:01.0: SPAN 1: Primary Sync Source</font></div>
<div><font face="Courier New">[ 17.708056] wct4xxp 0000:81:01.0: RCLK source set to span 1</font></div>
<div><font face="Courier New">[ 17.708065] wct4xxp 0000:81:01.0: Recovered timing mode, RCLK set to span 1</font></div>
<div><font face="Courier New">[ 17.736138] wct4xxp 0000:81:01.0: Span 2 configured for ESF/B8ZS</font></div>
<div><font face="Courier New">[ 17.808065] wct4xxp 0000:81:01.0: RCLK source set to span 1</font></div>
<div><font face="Courier New">[ 17.808073] wct4xxp 0000:81:01.0: Recovered timing mode, RCLK set to span 1</font></div>
<div><font face="Courier New">[ 17.864134] wct4xxp 0000:81:01.0: Span 3 configured for ESF/B8ZS</font></div>
<div><font face="Courier New">[ 17.908049] wct4xxp 0000:81:01.0: RCLK source set to span 1</font></div>
<div><font face="Courier New">[ 17.908058] wct4xxp 0000:81:01.0: Recovered timing mode, RCLK set to span 1</font></div>
<div><font face="Courier New">[ 17.992139] wct4xxp 0000:81:01.0: Span 4 configured for ESF/B8ZS</font></div>
<div><font face="Courier New">[ 18.008106] wct4xxp 0000:81:01.0: RCLK source set to span 1</font></div>
<div><font face="Courier New">[ 18.008114] wct4xxp 0000:81:01.0: Recovered timing mode, RCLK set to span 1</font></div>
<div><font face="Courier New">[ 20.208172] wct4xxp 0000:81:01.0: Setting yellow alarm span 1</font></div>
<div><font face="Courier New">[ 20.208212] wct4xxp 0000:81:01.0: RCLK source set to span 2</font></div>
<div><font face="Courier New">[ 20.208216] wct4xxp 0000:81:01.0: System timing mode, RCLK set to span 2</font></div>
<div><font face="Courier New">[ 20.308149] wct4xxp 0000:81:01.0: Setting yellow alarm span 2</font></div>
<div><font face="Courier New">[ 20.308180] wct4xxp 0000:81:01.0: RCLK source set to span 3</font></div>
<div><font face="Courier New">[ 20.308184] wct4xxp 0000:81:01.0: System timing mode, RCLK set to span 3</font></div>
<div><font face="Courier New">[ 20.408173] wct4xxp 0000:81:01.0: Setting yellow alarm span 3</font></div>
<div><font face="Courier New">[ 20.408200] wct4xxp 0000:81:01.0: RCLK source set to span 4</font></div>
<div><font face="Courier New">[ 20.408204] wct4xxp 0000:81:01.0: System timing mode, RCLK set to span 4</font></div>
<div><font face="Courier New">[ 25.601523] wct4xxp 0000:81:01.0: Span 1 configured for ESF/B8ZS</font></div>
<div><font face="Courier New">[ 25.601587] wct4xxp 0000:81:01.0: SPAN 1: Primary Sync Source</font></div>
<div><font face="Courier New">[ 25.601673] wct4xxp 0000:81:01.0: Span 4 configured for ESF/B8ZS</font></div>
<div><font face="Courier New">[ 25.608209] wct4xxp 0000:81:01.0: RCLK source set to span 4</font></div>
<div><font face="Courier New">[ 25.608215] wct4xxp 0000:81:01.0: System timing mode, RCLK set to span 4</font></div>
</div>
<div><br>
</div>
<div>Checking /proc/interrupts reveals that the card generated 100,000 interrupts without being serviced and the kernel disabled it (and also reveals that the card is apparently on its own IRQ):</div>
<div><br>
</div>
<div>
<div><font face="Courier New">maintenance@sip:~$ cat /proc/interrupts</font></div>
<div><font face="Courier New"> CPU0 CPU1</font></div>
<div><font face="Courier New"> 0: 46 0 IO-APIC-edge timer</font></div>
<div><font face="Courier New"> 1: 10 0 IO-APIC-edge i8042</font></div>
<div><font face="Courier New"> 7: 1 0 IO-APIC-edge</font></div>
<div><font face="Courier New"> 8: 0 0 IO-APIC-edge rtc0</font></div>
<div><font face="Courier New"> 9: 0 0 IO-APIC-fasteoi acpi</font></div>
<div><font face="Courier New"> 12: 4 0 IO-APIC-edge i8042</font></div>
<div><font face="Courier New"> 14: 0 0 IO-APIC-edge pata_amd</font></div>
<div><font face="Courier New"> 15: 0 0 IO-APIC-edge pata_amd</font></div>
<div><font face="Courier New"> 16: 304 0 IO-APIC-fasteoi nouveau</font></div>
<div><font face="Courier New"> 19: 1221 0 IO-APIC-fasteoi eth1</font></div>
<div><font face="Courier New"> 21: 8681 0 IO-APIC-fasteoi sata_nv</font></div>
<div><font face="Courier New"> 22: 0 0 IO-APIC-fasteoi ehci_hcd:usb1</font></div>
<div><font face="Courier New"> 23: 0 0 IO-APIC-fasteoi ohci_hcd:usb2</font></div>
<div><font face="Courier New"><b> 25: 100000 1 IO-APIC-fasteoi wct4xxp</b></font></div>
<div><font face="Courier New">NMI: 1 1 Non-maskable interrupts</font></div>
<div><font face="Courier New">LOC: 17884 19728 Local timer interrupts</font></div>
<div><font face="Courier New">SPU: 0 0 Spurious interrupts</font></div>
<div><font face="Courier New">PMI: 1 1 Performance monitoring interrupts</font></div>
<div><font face="Courier New">IWI: 1554 815 IRQ work interrupts</font></div>
<div><font face="Courier New">RTR: 0 0 APIC ICR read retries</font></div>
<div><font face="Courier New">RES: 6566 8577 Rescheduling interrupts</font></div>
<div><font face="Courier New">CAL: 220 4521 Function call interrupts</font></div>
<div><font face="Courier New">TLB: 638 504 TLB shootdowns</font></div>
<div><font face="Courier New">TRM: 0 0 Thermal event interrupts</font></div>
<div><font face="Courier New">THR: 0 0 Threshold APIC interrupts</font></div>
<div><font face="Courier New">MCE: 0 0 Machine check exceptions</font></div>
<div><font face="Courier New">MCP: 1 1 Machine check polls</font></div>
<div><font face="Courier New">ERR: 1</font></div>
<div><font face="Courier New">MIS: 0</font></div>
</div>
<div><br>
</div>
<div>Any ideas on how I can further diagnose and pursue this? Google does not reveal much related to this issue that is useful.</div>
<div><br>
</div>
<div>Thank you!</div>
<div><br>
<div apple-content-edited="true">
<div>--</div>
<div>Scott L. Lykens</div>
<div>Keystone Medical Management Solutions, Inc.</div>
<div>+1 814 325-7500 x501 -- <a href="http://www.kmmsinc.com">www.kmmsinc.com</a></div>
</div>
<br>
</div>
</body>
</html>