[Asterisk-bsd] freebsd panics with wct4xxp driver

Hooman Fazaeli hoomanfazaeli at gmail.com
Tue Jul 12 16:39:24 CDT 2016 

Hi,

We are working to setup an asterisk-based PBX in the office. Everything worked
until we decided to deploy ISDN PRI E1 links with the following hw/sw configuration:

Hardware:
     - Intel Core i5 + 4G RAM
- Digium Wildcard TE405P (2nd Gen) (4 TE span PCI-e PRI card)
Software:
     - FreeBSD 9.3-RELEASE i386 (kernel mostly GENERIC with some extra options/devices)
     - Asterisk 1.8.30.0
     - dahdi-2.4.0rc5_6 (from ports)
     - dahdi-kmod26-2.6.1.r10738 (from ports)

The problem is that we have occasional panics and the back-traces indicate that the
crashes are triggered by wct4xxp driver. Until now, I was able to catch
two of the panics:

First backtrace (spin lock held too long)
==========================================
#0  doadump (textdump=1) at pcpu.h:250
#1  0xc0ade835 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:454
#2  0xc0adeb32 in panic (fmt=<value optimized out>) at ../../../kern/kern_shutdown.c:642
#3  0xc0ac9cff in _mtx_lock_spin_failed (m=0x0) at ../../../kern/kern_mutex.c:515
#4  0xc0ac9e75 in _mtx_lock_spin (m=0xc90cdc10, tid=3385882832, opts=0,
     file=0xc90b060e "/usr/ports/misc//dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi"..., line=9415)
     at ../../../kern/kern_mutex.c:557
#5  0xc0aca691 in _mtx_lock_spin_flags (m=0xc90cdc10, opts=0,
     file=0xc90b060e "/usr/ports/misc//dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi"..., line=9415)
     at ../../../kern/kern_mutex.c:259
#6  0xc90ac65a in _process_masterspan ()
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi-base.c:9415
#7  0xc90ac7e2 in _dahdi_receive (span=0xc85df03c)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi-base.c:10064
#8  0xc936cff5 in __receive_span (ts=0xc85df000)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/wct4xxp/../../drivers/dahdi/wct4xxp/base.c:3134
#9  0xc936d072 in t4_prep_gen2 (wc=0xc8f7d400)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/wct4xxp/../../drivers/dahdi/wct4xxp/base.c:3169
#10 0xc936fd41 in _t4_interrupt_gen2 (dev_id=0xc8f7d400)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/wct4xxp/../../drivers/dahdi/wct4xxp/base.c:4233
#11 0xc0aaef07 in intr_event_handle (ie=0xc7d9c800, frame=0xf0ed6a30) at ../../../kern/kern_intr.c:1446
#12 0xc0fe42d9 in intr_execute_handlers (isrc=0xc7d8de84, frame=0xf0ed6a30) at ../../../x86/x86/intr_machdep.c:266
#13 0xc0fe7376 in lapic_handle_intr (vector=50, frame=0xf0ed6a30) at ../../../x86/x86/local_apic.c:780
#14 0xc0fb6455 in Xapic_isr1 () at apic_vector.s:89
#15 0xc0b2ba78 in selrecord (selector=0xc9d078d0, sip=0xc8588818) at atomic.h:258
#16 0xc909da69 in dahdi_device_poll (dev=0xc8f87100, events=3, td=0xc9d078d0)
     at /usr/ports/misc//dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi-base.c:10213
#17 0xc09b2364 in devfs_poll_f (fp=0xc9b10460, events=3, cred=0xc9617680, td=0xc9d078d0) at ../../../fs/devfs/devfs_vnops.c:1155
---Type <return> to continue, or q <return> to quit---
#18 0xc0b2bddb in sys_poll (td=0xc9d078d0, uap=0xf0ed6ccc) at file.h:319
#19 0xc0fcc8c3 in syscall (frame=0xf0ed6d08) at subr_syscall.c:135
#20 0xc0fb60f1 in Xint0x80_syscall () at ../../../i386/i386/exception.s:270
#21 0x00000033 in ?? ()

Backtrace 2 (page fault)
========================
#0  doadump (textdump=1) at pcpu.h:250
#1  0xc0ade835 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:454
#2  0xc0adeb32 in panic (fmt=<value optimized out>) at ../../../kern/kern_shutdown.c:642
#3  0xc0fcbfd3 in trap_fatal (frame=0xc7bc4a38, eva=3451666600) at ../../../i386/i386/trap.c:1036
#4  0xc0fcc047 in trap_pfault (frame=0xc7bc4a38, usermode=0, eva=3451666600) at ../../../i386/i386/trap.c:851
#5  0xc0fcd479 in trap (frame=0xc7bc4a38) at ../../../i386/i386/trap.c:547
#6  0xc0fb605c in calltrap () at ../../../i386/i386/exception.s:170
#7  0xc929e1ce in __dahdi_transmit_chunk (chan=0xc95b5000, buf=0xc804a110 "ÕÕÕÕÕÕÕÕ", 'ÿ' <repeats 104 times>, "óóóóóóóó", 'ÿ' <repeats 80 times>...)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi-base.c:7989
#8  0xc92a4ea7 in _dahdi_transmit (span=0xc857503c)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/dahdi/../../drivers/dahdi/dahdi-base.c:9527
#9  0xc955e0e4 in __transmit_span (ts=0xc8575000)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/wct4xxp/../../drivers/dahdi/wct4xxp/base.c:3139
#10 0xc955e084 in t4_prep_gen2 (wc=0xc85b2300)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/wct4xxp/../../drivers/dahdi/wct4xxp/base.c:3170
#11 0xc9560d41 in _t4_interrupt_gen2 (dev_id=0xc85b2300)
     at /usr/ports/misc/dahdi-kmod26/work/dahdi-freebsd-2.6.1-r10738/bsd-kmod/wct4xxp/../../drivers/dahdi/wct4xxp/base.c:4233
#12 0xc0aaef07 in intr_event_handle (ie=0xc7d9c800, frame=0xc7bc4c0c) at ../../../kern/kern_intr.c:1446
#13 0xc0fe42d9 in intr_execute_handlers (isrc=0xc7d8de84, frame=0xc7bc4c0c) at ../../../x86/x86/intr_machdep.c:266
#14 0xc0fe7376 in lapic_handle_intr (vector=50, frame=0xc7bc4c0c) at ../../../x86/x86/local_apic.c:780
#15 0xc0fb6455 in Xapic_isr1 () at apic_vector.s:89
#16 0xc0fbbf0c in cpu_idle_mwait (busy=1) at cpufunc.h:154
#17 0xc0fbee10 in cpu_idle (busy=1) at ../../../i386/i386/machdep.c:1360
#18 0xc0b08c06 in sched_idletd (dummy=0x0) at ../../../kern/sched_ule.c:2609
#19 0xc0aaa96f in fork_exit (callout=0xc0b088a0 <sched_idletd>, arg=0x0, frame=0xc7bc4d08) at ../../../kern/kern_fork.c:996
#20 0xc0fb6104 in fork_trampoline () at ../../../i386/i386/exception.s:279

The panics seem to have direct relation with load. However, as we are on the test phase, the
system load has been very low at the time of panics.

Does anybody else has experienced such/similar panics? Any suggestion as how to fix them?
I would also like to hear of PRI configurations (HW+DAHDI+Asterisk+PRIcard) on FreeBSD which
has stably worked in production.

Please keep me CC'd and let me know if you need more information
about the system and panics.

Thanks in advance.

-- 
Best regards
Hooman Fazaeli

More information about the Asterisk-BSD mailing list