Intel DQ77KB: high interrupt rate when using HDMI

patpro

Active Member

Reaction score: 15
Messages: 211

Hello,

I've bought a brand new PC to act as gateway/firewall/file server/... at home. It's based upon an Intel DQ77KB motherboard, Core i3, Intel Centrino Advanced-N 6230. It boots on a mini-pci SSD (Intel SSDMCEAC060B3), and I've added 2 SATA HDD (HGST HTS721010A9E630 JB0OA3B0) on SATA6 ports.

It's my first install of FreeBSD 9.1 - fully updated (I'm used to 8.x). It's also my first full ZFS install.

For now, the box is still a work in progress, and I've few problems with it. Biggest one is the interrupt rate becoming quite huge as soon as the state of the HDMI port changes (plug or unplug, switch off display, switch display to use another video input...).
When the HDMI port change states, [CMD=""]top -S[/CMD] shows that [CMD=""]intr[/CMD] goes from 0% and stick around 65-70%. CPU load average climbs to 0.8-0.9. It's a fanless box, so it means also higher temperature...

[CMD=""]vmstat -i[/CMD] shows that the problem comes from IRQ 16 (xhci0 and ehci0). Here is a "normal" output, before triggering the problem:
Code:
$ vmstat -i
interrupt                          total       rate
irq16: xhci0 ehci0                150863         63
irq23: ehci1                        4923          2
cpu0:timer                        134646         56
irq265: hdac0                         80          0
irq266: iwn0                       49541         20
irq267: em1:rx 0                    3771          1
irq268: em1:tx 0                    1066          0
irq269: em1:link                       2          0
irq270: ahci0                       4417          1
cpu1:timer                         30750         12
cpu2:timer                         23486          9
cpu3:timer                         38270         16
Total                             441815        186
Same command just after an HDMI cable is plugged:
Code:
$ vmstat -i
interrupt                          total       rate
irq16: xhci0 ehci0               6468138       2552
irq23: ehci1                        5250          2
cpu0:timer                        142452         56
irq265: hdac0                         80          0
irq266: iwn0                       53391         21
irq267: em1:rx 0                    3972          1
irq268: em1:tx 0                    1091          0
irq269: em1:link                       2          0
irq270: ahci0                       4587          1
cpu1:timer                         32544         12
cpu2:timer                         25519         10
cpu3:timer                         86331         34
Total                            6823357       2692
and the corresponding top output:

Code:
$ top -SHb 1000 | grep "intr{irq"
   12 root     -88    -     0K   304K CPU1    3   0:55 68.46% intr{irq16: xhci0 ehc}
   12 root     -88    -     0K   304K WAIT    1   0:00  0.00% intr{irq270: ahci0}
   12 root     -92    -     0K   304K WAIT    0   0:00  0.00% intr{irq267: em1:rx 0}
   12 root     -92    -     0K   304K WAIT    3   0:00  0.00% intr{irq266: iwn0}
   12 root     -88    -     0K   304K WAIT    0   0:00  0.00% intr{irq23: ehci1}
   12 root     -92    -     0K   304K WAIT    0   0:00  0.00% intr{irq268: em1:tx 0}
   12 root     -92    -     0K   304K WAIT    2   0:00  0.00% intr{irq269: em1:link}
   12 root     -96    -     0K   304K WAIT    2   0:00  0.00% intr{irq265: hdac0}
The only solution to reset this behavior is to reboot the system.

What can I do to prevent this interrupt problem? I've though about disabling xHCI, but I can't find a way to prevent the driver to load. I've added
Code:
xhci_load="NO"
into /boot/loader.conf, but no effect.

dmesg: http://patpro.net/dmesg_DQ77KB.txt
[CMD=]kldstat -v[/CMD] output: http://patpro.net/kldstat_DQ77KB.txt
 

cpm@

Moderator
Staff member
Moderator
Developer

Reaction score: 947
Messages: 2,140

Simply disabling USB 3.0 support in the BIOS (in case you have that option) or try setting
Code:
hint.xhci.0.disabled="1"
in /boot/device.hints. See device.hints(5) for more information.
 
OP
patpro

patpro

Active Member

Reaction score: 15
Messages: 211

Hello,

BIOS is not a very nice option, I can't find any option to downgrade ports from USB3 to USB2, I can only disable USB ports. Shutting down every USB3 capable ports would leave me with 2 ports instead of 6. I'm going to give device.hints a try.

Any idea why playing with HDMI ports would trigger xHCI interrupt mini-storm?

edit: the device.hints trick does not work :/ after reboot the driver is loaded:

Code:
$ dmesg | grep xhci
xhci0: <Intel Panther Point USB 3.0 controller> mem 0xf7e20000-0xf7e2ffff irq 16 at device 20.0 on pci0
xhci0: 32 byte context size.
usbus0 on xhci0
 
OP
patpro

patpro

Active Member

Reaction score: 15
Messages: 211

After compiling a kernel without xHCI and rebooting, high interrupt still occurs when (un)plugging HDMI cable:

Code:
$ vmstat -i
interrupt                          total       rate
irq16: ehci0                    43507710     100712
irq23: ehci1                         920          2
cpu0:timer                         39003         90
irq265: hdac0                         83          0
irq266: iwn0                        7280         16
irq267: em1:rx 0                     693          1
irq268: em1:tx 0                     200          0
irq269: em1:link                       2          0
irq270: ahci0                       2731          6
cpu1:timer                          6935         16
cpu2:timer                          4282          9
cpu3:timer                        244247        565
Total                           43814086     101421
Interrupt mini-storm is not as high as with xHCI:

Code:
  PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
   11 root     155 ki31     0K    64K RUN     0   7:59 100.00% idle{idle: cpu0}
   11 root     155 ki31     0K    64K CPU2    2   7:55 100.00% idle{idle: cpu2}
   11 root     155 ki31     0K    64K CPU1    1   7:54 100.00% idle{idle: cpu1}
   11 root     155 ki31     0K    64K RUN     3   6:27 62.70% idle{idle: cpu3}
   12 root     -88    -     0K   304K WAIT    3   1:28 41.70% intr{irq16: ehci0}
I don't understand the link between this IRQ 16 and the HDMI port. I'll try the display port as soon as I find an adapter, but I'm not very confident it can resolve my problem.
 

cpm@

Moderator
Staff member
Moderator
Developer

Reaction score: 947
Messages: 2,140

Sorry for my previous post, out of issue ;)

xhci(4) has this tunables sysctl:
Code:
static SYSCTL_NODE(_hw_usb, OID_AUTO, xhci, CTLFLAG_RW, 0, "USB XHCI");
SYSCTL_INT(_hw_usb_xhci, OID_AUTO, debug, CTLFLAG_RW | CTLFLAG_TUN,
    &xhcidebug, 0, "Debug level");
TUNABLE_INT("[FILE]hw.usb.xhci.debug[/FILE]", &xhcidebug);
SYSCTL_INT(_hw_usb_xhci, OID_AUTO, xhci_port_route, CTLFLAG_RW | CTLFLAG_TUN,
    &xhciroute, 0, "Routing bitmap for switching EHCI ports to XHCI controller");
TUNABLE_INT("[FILE]hw.usb.xhci.xhci_port_route[/FILE]", &xhciroute);
#endif
Well, hw.usb.xhci.xhci_port_route variable decides if a port is connected to the EHCI or xHCI.

Take a look to loader tunables for the ehci(4) driver. Try add
Code:
hw.usb.ehci.no_hs="1"
to /boot/loader.conf and check if vmstat output shows a very high interrupt rate on irq16 again.
 
OP
patpro

patpro

Active Member

Reaction score: 15
Messages: 211

I'm not so sure it belongs here. I have absolutely no problem with USB, so far. My problem is triggered by the HDMI connector.

Looks like going with "modern" hardware for a FreeBSD router was not such a great idea. I should have stuck with a Supermicro or Tyan motherboard.

- edit -
Thanks CPU82 for the update. I'll give this a try soon.
 

cpm@

Moderator
Staff member
Moderator
Developer

Reaction score: 947
Messages: 2,140

If doesn't solves your problem, feel free to submit a PR to freebsd-current@ ML.

Compile your kernel with:
Code:
options USB_DEBUG
Then enable after boot:
Code:
# sysctl hw.usb.ehci.debug=15 
# sleep 1 
# sysctl hw.usb.ehci.debug=0
# tail -n 100 /var/log/messages | tee hw.usb.debug.txt
Add hw.usb.debug.txt to your PR to make work more easy to developers.
 
OP
patpro

patpro

Active Member

Reaction score: 15
Messages: 211

No luck with hw.usb.ehci.no_hs… I'll submit a PR in few days, my "play time" is exhausted for now.
 
Top