[Asterisk-Dev] Debugging Help - odd behaviour from wctdm driver

[Richard Scobie]

I have been running two Asterisks for almost 2 years now, using TDM400P 
and X100P cards, and while perfomance has been good, I would really like 
to get to the bottom of intermittent problems (1-3 month driver 
reloads/reboots to restore operation). Reliability has dropped somewhat 
since X100Ps were replaced by TDM400P FXO modules.

The systems are very simple - analog only, no SIP etc., G711 ulaw codec 
only, no voicemail, conferencing or MOH and the 2 systems are IAX2-ed 
together.

Both boxes have a very minimal Redhat 7.3 install, with a custom 
compiled 2.4.27 kernel containing only what is necessary for Asterisk. 
The only non system processes other than asterisk running, are ntpd, 
xinetd, a couple of mingettys and sshd. Asterisk and Zaptel are CVS 
updated on a weekly basis.

Hardware-wise, both are identical P4 2.4, Intel 845 chipset machines, 
the only difference is one has a single TDM400 with 4 x FXO and the 
other has 1 x TDM400 with 2 x FXO and 2 x TDM400, each with 4 FXS.

Sorry for the long preamble - here is observation I need assistence with.

With Asterisk not running, and just the wctdm driver loaded and no 
activity on the machine, the output of "vmstat 1" shows this on the box 
with only one TDM card:

   system      cpu
    in    cs  us  sy  id
  1113     5   0   0 100
  1114     7   0  52  48
  1113     7   0 100   0
  1114     5   0 100   0
  1113     5   0 100   0
  1113     5   0  10  90
  1113     7   0   0 100
  1113     7   0   0 100
  1113     5   0   0 100
  1113     5   0   0 100
  1113     5   0   0 100
  1113     7   0   0 100
  1117     7   0   0 100
  1113     5   0   0 100
  1113     5   0   0 100
  1113     5   0   0 100
  1113     7   0   0 100
  1113     7   0   0 100
  1113     5   0   0 100
  1114     5   0   0 100
  1112     7   0   0 100
  1113     7   0   0 100
  1112     8   0   0 100
  1113     7   0   0 100
  1113     5   0   0 100
  1113     5   0  29  71
  1113     7   0 100   0
  1113     7   0 100   0
  1113     5   0 100   0
  1113     5   0  32  68
  1114     5   0   0 100
  1113     7   0   0 100
  1113     7   0   0 100
  1113     5   0   0 100

This CPU usage pattern repeats at the same interval.

The other box with 3 TDM400s shows:

   system      cpu
    in    cs  us  sy  id
  3123     5   0   0 100
  3135    14   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100
  3133     5   0   9  91
  3133     8   0  50  50
  3133     5   0  50  50
  3133     5   0  50  50
  3133     5   0  50  50
  3133     5   0  50  50
  3145     8   0  50  50
  3133     5   0  50  50
  3133     5   0  48  52
  3133     5   0   0 100
  3133     5   0   0 100
  3133     8   0   0 100
  3137     5   0   0 100
  3132     5   0   0 100
  3134     5   0   0 100
  3132     5   0   0 100
  3134     8   0   0 100
  3132     6   0   0 100
  3134     5   0   0 100
  3132     5   0   0 100
  3134     5   0   0 100
  3132     8   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100
  3149     9   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100
  3133     5   0   2  98
  3133     5   0   2  98
  3133     8   0   1  99
  3133     5   0   2  98
  3133     5   0  80  20
  3133     5   0 100   0
  3133     5   0 100   0
  3133     8   0  26  74
  3133     5   0   0 100
  3133     5   0   0 100
  3133     5   0   0 100

Again, this CPU load pattern repeats at the same interval, alternating 
bursts of 50% sys CPU and shorter bursts of 100% sys.

This behaviour only occurs with the wctdm driver loaded. Perhaps it is 
quite normal behaviour, but I am wondering if there is some sort of race 
occuring in the driver.

While having many years of Linux experience, I am unfortunately not a 
developer and wonder if someone can please tell me how I would go about 
finding which function in the driver is eating all the CPU?

Thanks for any assistence.

Richard