Interrupts Affinity on ARM systems

[FreebsdRider]

I'm using the same FreeBSD kernel on ARM based instance and Intel based instance.
ARM based instance uname:

uname -a
FreeBSD freebsd 13.2-RELEASE FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC arm64

Intel based instance uname:

uname -a
FreeBSD freebsd 13.2-RELEASE FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC amd64

For the Intel based instance, I want to see the affinity of the hardware interrupts so I run the following:

 $ vmstat -i
interrupt                          total       rate
.
.
irq41: ena0                          137          4
irq42: ena0                           97          3
irq43: ena0                           18          1
irq44: ena0                         5254        159

Now after I run cpuset I get:

$ cpuset -g -x 44
irq 44 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15

However on ARM instance I do the same and get the following:

$ vmstat -i
interrupt                                             total       rate
gic0,p11:-ic_timer0                                44021813        503
..
its0,75: ena0                                           580          0
its0,76: ena0                                          2943          0
its0,77: ena0                                           586          0

After running cpuset:

 $ cpuset -g -x 75
cpuset: getaffinity: No such process

Why are the interrupts name are different in ARM/Intel and how can I get the interrupts affinity in ARM systems?

Followup question:
Upon printing the cpu on which the HW interrupts are received (curcpu parameter within the msix interrupt handler) on ARM systems I can see that all interrupts are received on CPU 0. Why is that?
Is it an acceptable practice in FreeBSD to spread the interrupt affinity upon different cores by default using bus_bind_intr() calls?

 

[ulope]

I have the same question.

I run OPNSense (based on FreeBSD 13) on a Rockchip RK3399 based board and would very much like to bind the network interrupts to the high performance cores.

root@gw:~ # vmstat -i | egrep 'dwc0|re0'
gic0,s12: dwc0                                      4242350         66
its0,0: re0                                         5430746         84

root@gw:~ # cpuset -g -x 12
cpuset: getaffinity: No such process
root@gw:~ # cpuset -g -x s12
cpuset: getaffinity: No such process
root@gw:~ # cpuset -g -x gic0,s12
cpuset: getaffinity: No such process

 

[FreebsdRider]

I have the same question.

I run OPNSense (based on FreeBSD 13) on a Rockchip RK3399 based board and would very much like to bind the network interrupts to the high performance cores.

root@gw:~ # vmstat -i | egrep 'dwc0|re0'
gic0,s12: dwc0                                      4242350         66
its0,0: re0                                         5430746         84

root@gw:~ # cpuset -g -x 12
cpuset: getaffinity: No such process
root@gw:~ # cpuset -g -x s12
cpuset: getaffinity: No such process
root@gw:~ # cpuset -g -x gic0,s12
cpuset: getaffinity: No such process

As far as my short experience with ARM based fbsd, I was able to bind interrupts to cpus only through changing the network driver code by adding bus_bind_intr() after the interrupts were allocated.
It seems like freebsd on ARM is still being developed

 

[ulope]

Alright I figured out a way to set the affinity without recompiling the driver.

Apparently the problem with cpuset is that it can't figure out the mapping between interrupts and processes on ARM.

But when you manually look up the corresponding process via ps cpuset works as intended.

Example:

# This should result in one process that handles all the interrupts
ps faxH | grep intr

# In my case CPU 4 and 5 are the high performance cores
cpuset -p <pid from above> -l 4,5