MSI/MSI-X on Intel em NIC

wrkilu · #1 November 15th, 2011, 14:02

Hi,

We have a border router with 5 em nics. OS is FreeBSD 8.2 x64.

Code:

# top -PSH
last pid:  6097;  load averages:  1.46,  1.45,  1.48                                                    
                                                      up 31+13:32:57  14:02:32
145 processes: 11 running, 111 sleeping, 23 waiting
CPU 0:  0.0% user,  0.0% nice,  0.0% system, 30.3% interrupt, 69.7% idle
CPU 1:  0.4% user,  0.0% nice,  0.4% system,  4.5% interrupt, 94.7% idle
CPU 2:  0.0% user,  0.0% nice,  3.8% system, 10.5% interrupt, 85.7% idle
CPU 3:  0.0% user,  0.0% nice, 46.8% system,  0.4% interrupt, 52.8% idle
CPU 4:  0.0% user,  0.0% nice, 29.6% system,  0.0% interrupt, 70.4% idle
CPU 5:  0.4% user,  0.0% nice, 14.7% system,  0.0% interrupt, 85.0% idle
CPU 6:  0.0% user,  0.0% nice, 64.8% system,  0.0% interrupt, 35.2% idle
CPU 7:  0.0% user,  0.0% nice,  0.0% system,  4.9% interrupt, 95.1% idle
Mem: 374M Active, 2576M Inact, 782M Wired, 31M Cache, 412M Buf, 91M Free
Swap: 10G Total, 10G Free

  PID USERNAME   PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
   11 root       171 ki31     0K   128K CPU7    7 715.5H 100.00% {idle: cpu7}
   11 root       171 ki31     0K   128K CPU1    1 719.5H 96.53% {idle: cpu1}
   11 root       171 ki31     0K   128K RUN     5 680.7H 86.87% {idle: cpu5}
   11 root       171 ki31     0K   128K CPU2    2 708.4H 78.71% {idle: cpu2}
   11 root       171 ki31     0K   128K CPU4    4 510.5H 72.71% {idle: cpu4}
   11 root       171 ki31     0K   128K CPU0    0 549.6H 71.92% {idle: cpu0}
    0 root       -68    0     0K   208K -       6 380.0H 65.97% {em4 taskq}
   11 root       171 ki31     0K   128K CPU3    3 466.1H 57.86% {idle: cpu3}
    0 root       -68    0     0K   208K CPU3    3 290.3H 45.26% {em1 taskq}
   11 root       171 ki31     0K   128K CPU6    6 384.3H 39.40% {idle: cpu6}
    0 root       -68    0     0K   208K CPU4    4 245.6H 33.84% {em2 taskq}
   12 root       -68    -     0K   368K WAIT    0 204.1H 31.64% {irq256: em0:rx 0}
   12 root       -44    -     0K   368K WAIT    3  81.7H 25.49% {swi1: netisr 0}
    0 root       -68    0     0K   208K -       5  53.9H 14.94% {em3 taskq}
   12 root       -68    -     0K   368K WAIT    1 506:54  1.22% {irq257: em0:tx 0}
    0 root       -68    0     0K   208K -       7  26.5H  0.78% {dummynet}
14434 root        44    0 26984K  5800K select  3 279:09  0.63% snmpd
   12 root       -32    -     0K   368K WAIT    1 243:50  0.00% {swi4: clock}
    0 root       -68    0     0K   208K -       4 139:00  0.00% {em0 txq}
   13 root        44    -     0K    16K -       4  85:22  0.00% yarrow
    0 root       -68    0     0K   208K -       1  64:54  0.00% {em0 rxq}

Hot to turn on MSI on these cards? How to check is it enabled?

**wblock@** · #2 November 15th, 2011, 15:52

Check /var/log/messages:

Code:

em0: Using MSIX interrupts with 3 vectors

I thought there was a sysctl to disable MSI, but looking through the output of sysctl -a I don't see one for em(4).

**phoenix** · #3 November 15th, 2011, 20:18

It all depends on the version of the em(4) driver, the version of the NIC chipset, and the version of FreeBSD in use. For example, on my 9.0-RC2 box, I have the following sysctl (there's no em-based NIC in this box, though):

Code:

hw.em.enable_msix: 1

On my 8.2-STABLE (r224019) box with this NIC:

Code:

em0@pci0:9:0:0: class=0x020000 card=0x10bc8086 chip=0x10bc8086 rev=0x06 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82571EB Gigabit Ethernet Controller (Copper)'
    class      = network
    subclass   = ethernet

I don't have that sysctl.

wrkilu · #4 November 16th, 2011, 11:08

Well, we've found the reason, Only em0 on our router supports MSI-X and only she has few interrupts on few cores. In Freebsd 8.2 that feature is enabled in default.

Code:

# pciconv -lcv
em0@pci0:1:0:0: class=0x020000 card=0x34da8086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
    class      = network
    subclass   = ethernet
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit
    cap 10[e0] = PCI-Express 1 endpoint max data 256(256) link x1(x1)
    cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected


em1@pci0:2:0:0: class=0x020000 card=0x10828086 chip=0x107d8086 rev=0x06 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'PRO/1000 PT'
    class      = network
    subclass   = ethernet
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
    cap 10[e0] = PCI-Express 1 endpoint max data 256(256) link x1(x1)

regards

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Quakelive.com / Firefox / .msi file	ZombieBeefchunk	Installation and Maintenance of FreeBSD Ports or Packages	6	August 2nd, 2012 18:36
[Solved] Wine no more msi?	mdma	Installation and Maintenance of FreeBSD Ports or Packages	3	July 9th, 2009 13:31
Running FreeBSD on MSI Wind U100	giannidoe	Mobile Computing	3	May 25th, 2009 02:25
MSI motherboard?	xman73	System Hardware	7	November 28th, 2008 14:05
MSI Wind U90+wi-fi+freebsd	sT4k3	Mobile Computing	0	November 24th, 2008 13:17