Hi!
I have a server:
I did a little tuning:
Each user interface Intel X520-DA2 provides 8 queues:
The server is used as a border router - 2 full viev.
Each Intel Xeon E5-2620 has 6 cores - a total of 12 cores, Intel X650 DA2 with two interfaces 16 queues. As a result, part of the cores remain idle:
How I can configure the binding queues ix0 only CPU1 cores, and queues ix1 only CPU2 cores (eg reducing the number of queues to 6 on each interface)?
I have a server:
Code:
CPU: 2x Intel (R) Xeon (R) CPU E5-2620 0@2.00GHz (2000.01-MHz K8-class CPU) (HT off)
RAM: 8GB
NIC: Intel Server Adapter X520-DA2 (2 link x10G to swith)
FreeBSD 8.3-RELEASE-p4I did a little tuning:
Code:
# cat /boot/loader.conf
...
hw.ixgbe.rxd=4096
hw.ixgbe.txd=4096
...
# cat /etc/sysctl.conf
...
# NMBClusters (default 25600)
kern.ipc.nmbclusters=512000
# JumboClusters (default 12800)
kern.ipc.nmbjumbop=256000
# Intel recomend increase (default 1000)
hw.intr_storm_threshold=9000
# IXGB
dev.ix.0.rx_processing_limit=4096
dev.ix.1.rx_processing_limit=4096
...Each user interface Intel X520-DA2 provides 8 queues:
Code:
sysctl -a | grep ix.0
dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.4.5
dev.ix.0.%driver: ix
dev.ix.0.%location: slot=0 function=0
dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x0003 class=0x020000
dev.ix.0.%parent: pci6
dev.ix.0.fc: 3
dev.ix.0.advertise_gig: 0
dev.ix.0.enable_aim: 1
dev.ix.0.advertise_speed: 0
dev.ix.0.rx_processing_limit: 4096
dev.ix.0.dropped: 0
dev.ix.0.mbuf_defrag_failed: 0
dev.ix.0.no_tx_dma_setup: 0
dev.ix.0.watchdog_events: 0
dev.ix.0.tso_tx: 25255
dev.ix.0.link_irq: 6
dev.ix.0.queue0.interrupt_rate: 250000
dev.ix.0.queue0.txd_head: 2148
dev.ix.0.queue0.txd_tail: 2148
dev.ix.0.queue0.no_desc_avail: 0
dev.ix.0.queue0.tx_packets: 2548424586
dev.ix.0.queue0.rxd_head: 193
dev.ix.0.queue0.rxd_tail: 192
dev.ix.0.queue0.rx_packets: 2542892969
dev.ix.0.queue0.rx_bytes: 92154394802
dev.ix.0.queue0.lro_queued: 0
dev.ix.0.queue0.lro_flushed: 0
dev.ix.0.queue1.interrupt_rate: 250000
dev.ix.0.queue1.txd_head: 2645
dev.ix.0.queue1.txd_tail: 2645
dev.ix.0.queue1.no_desc_avail: 434
dev.ix.0.queue1.tx_packets: 2538223612
dev.ix.0.queue1.rxd_head: 824
dev.ix.0.queue1.rxd_tail: 823
dev.ix.0.queue1.rx_packets: 2520184131
dev.ix.0.queue1.rx_bytes: 76553875238
dev.ix.0.queue1.lro_queued: 0
dev.ix.0.queue1.lro_flushed: 0
dev.ix.0.queue2.interrupt_rate: 1000000
dev.ix.0.queue2.txd_head: 3042
dev.ix.0.queue2.txd_tail: 3044
dev.ix.0.queue2.no_desc_avail: 0
dev.ix.0.queue2.tx_packets: 2521877482
dev.ix.0.queue2.rxd_head: 666
dev.ix.0.queue2.rxd_tail: 662
dev.ix.0.queue2.rx_packets: 2497515347
dev.ix.0.queue2.rx_bytes: 68766948284
dev.ix.0.queue2.lro_queued: 0
dev.ix.0.queue2.lro_flushed: 0
dev.ix.0.queue3.interrupt_rate: 38461
dev.ix.0.queue3.txd_head: 1847
dev.ix.0.queue3.txd_tail: 1847
dev.ix.0.queue3.no_desc_avail: 21
dev.ix.0.queue3.tx_packets: 2511792679
dev.ix.0.queue3.rxd_head: 354
dev.ix.0.queue3.rxd_tail: 353
dev.ix.0.queue3.rx_packets: 2484006427
dev.ix.0.queue3.rx_bytes: 72600953889
dev.ix.0.queue3.lro_queued: 0
dev.ix.0.queue3.lro_flushed: 0
dev.ix.0.queue4.interrupt_rate: 22727
dev.ix.0.queue4.txd_head: 1199
dev.ix.0.queue4.txd_tail: 1199
dev.ix.0.queue4.no_desc_avail: 0
dev.ix.0.queue4.tx_packets: 2503894689
dev.ix.0.queue4.rxd_head: 632
dev.ix.0.queue4.rxd_tail: 628
dev.ix.0.queue4.rx_packets: 2494369894
dev.ix.0.queue4.rx_bytes: 68695300417
dev.ix.0.queue4.lro_queued: 0
dev.ix.0.queue4.lro_flushed: 0
dev.ix.0.queue5.interrupt_rate: 21276
dev.ix.0.queue5.txd_head: 976
dev.ix.0.queue5.txd_tail: 976
dev.ix.0.queue5.no_desc_avail: 0
dev.ix.0.queue5.tx_packets: 2492246991
dev.ix.0.queue5.rxd_head: 682
dev.ix.0.queue5.rxd_tail: 681
dev.ix.0.queue5.rx_packets: 2485693582
dev.ix.0.queue5.rx_bytes: 74135189471
dev.ix.0.queue5.lro_queued: 0
dev.ix.0.queue5.lro_flushed: 0
dev.ix.0.queue6.interrupt_rate: 1000000
dev.ix.0.queue6.txd_head: 3477
dev.ix.0.queue6.txd_tail: 3477
dev.ix.0.queue6.no_desc_avail: 0
dev.ix.0.queue6.tx_packets: 2521664630
dev.ix.0.queue6.rxd_head: 369
dev.ix.0.queue6.rxd_tail: 368
dev.ix.0.queue6.rx_packets: 2502188306
dev.ix.0.queue6.rx_bytes: 71013517996
dev.ix.0.queue6.lro_queued: 0
dev.ix.0.queue6.lro_flushed: 0
dev.ix.0.queue7.interrupt_rate: 100000
dev.ix.0.queue7.txd_head: 2415
dev.ix.0.queue7.txd_tail: 2419
dev.ix.0.queue7.no_desc_avail: 0
dev.ix.0.queue7.tx_packets: 2505236977
dev.ix.0.queue7.rxd_head: 30
dev.ix.0.queue7.rxd_tail: 29
dev.ix.0.queue7.rx_packets: 2486859634
dev.ix.0.queue7.rx_bytes: 71129476131
dev.ix.0.queue7.lro_queued: 0
dev.ix.0.queue7.lro_flushed: 0
dev.ix.0.mac_stats.crc_errs: 0
dev.ix.0.mac_stats.ill_errs: 0
dev.ix.0.mac_stats.byte_errs: 0
dev.ix.0.mac_stats.short_discards: 0
dev.ix.0.mac_stats.local_faults: 16
dev.ix.0.mac_stats.remote_faults: 6
dev.ix.0.mac_stats.rec_len_errs: 0
dev.ix.0.mac_stats.link_xon_txd: 0
dev.ix.0.mac_stats.link_xon_rcvd: 0
dev.ix.0.mac_stats.link_xoff_txd: 0
dev.ix.0.mac_stats.link_xoff_rcvd: 0
dev.ix.0.mac_stats.total_octets_rcvd: 15347000838325
dev.ix.0.mac_stats.good_octets_rcvd: 15346995346277
dev.ix.0.mac_stats.total_pkts_rcvd: 20033069037
dev.ix.0.mac_stats.good_pkts_rcvd: 20013548375
dev.ix.0.mac_stats.mcast_pkts_rcvd: 53207
dev.ix.0.mac_stats.bcast_pkts_rcvd: 810141
dev.ix.0.mac_stats.rx_frames_64: 1864192202
dev.ix.0.mac_stats.rx_frames_65_127: 6904972743
dev.ix.0.mac_stats.rx_frames_128_255: 726915375
dev.ix.0.mac_stats.rx_frames_256_511: 415966891
dev.ix.0.mac_stats.rx_frames_512_1023: 502229455
dev.ix.0.mac_stats.rx_frames_1024_1522: 9618756620
dev.ix.0.mac_stats.recv_undersized: 0
dev.ix.0.mac_stats.recv_fragmented: 0
dev.ix.0.mac_stats.recv_oversized: 0
dev.ix.0.mac_stats.recv_jabberd: 0
dev.ix.0.mac_stats.management_pkts_rcvd: 0
dev.ix.0.mac_stats.management_pkts_drpd: 0
dev.ix.0.mac_stats.checksum_errs: 270379702
dev.ix.0.mac_stats.good_octets_txd: 16000191922608
dev.ix.0.mac_stats.total_pkts_txd: 20143207467
dev.ix.0.mac_stats.good_pkts_txd: 20143207467
dev.ix.0.mac_stats.bcast_pkts_txd: 735
dev.ix.0.mac_stats.mcast_pkts_txd: 19718
dev.ix.0.mac_stats.management_pkts_txd: 0
dev.ix.0.mac_stats.tx_frames_64: 3444758841
dev.ix.0.mac_stats.tx_frames_65_127: 4981392057
dev.ix.0.mac_stats.tx_frames_128_255: 719059016
dev.ix.0.mac_stats.tx_frames_256_511: 424448807
dev.ix.0.mac_stats.tx_frames_512_1023: 492128975
dev.ix.0.mac_stats.tx_frames_1024_1522: 10081419771
dev.ix.0.mac_stats.fc_crc: 0
dev.ix.0.mac_stats.fc_last: 0
dev.ix.0.mac_stats.fc_drpd: 0
dev.ix.0.mac_stats.fc_pkts_rcvd: 0
dev.ix.0.mac_stats.fc_pkts_txd: 0
dev.ix.0.mac_stats.fc_dword_rcvd: 0
dev.ix.0.mac_stats.fc_dword_txd: 0The server is used as a border router - 2 full viev.
Each Intel Xeon E5-2620 has 6 cores - a total of 12 cores, Intel X650 DA2 with two interfaces 16 queues. As a result, part of the cores remain idle:
Code:
# # top -aSP
last pid: 30435; load averages: 2.11, 1.92, 2.00 up 1+07:32:26 13:50:08
57 processes: 2 running, 53 sleeping, 1 stopped, 1 waiting
CPU 0: 0.0% user, 0.0% nice, 1.4% system, 21.7% interrupt, 76.8% idle
CPU 1: 0.0% user, 0.0% nice, 1.4% system, 20.3% interrupt, 78.3% idle
CPU 2: 0.0% user, 0.0% nice, 1.4% system, 11.6% interrupt, 87.0% idle
CPU 3: 0.0% user, 0.0% nice, 2.9% system, 11.6% interrupt, 85.5% idle
CPU 4: 0.0% user, 0.0% nice, 0.0% system, 10.1% interrupt, 89.9% idle
CPU 5: 1.4% user, 0.0% nice, 0.0% system, 18.8% interrupt, 79.7% idle
CPU 6: 0.0% user, 0.0% nice, 4.3% system, 18.8% interrupt, 76.8% idle
CPU 7: 0.0% user, 0.0% nice, 1.4% system, 26.1% interrupt, 72.5% idle
CPU 8: 0.0% user, 0.0% nice, 1.4% system, 0.0% interrupt, 98.6% idle
CPU 9: 0.0% user, 0.0% nice, 1.4% system, 0.0% interrupt, 98.6% idle
CPU 10: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
CPU 11: 0.0% user, 0.0% nice, 1.4% system, 0.0% interrupt, 98.6% idle
Mem: 513M Active, 15M Inact, 587M Wired, 84K Cache, 76M Buf, 6732M Free
Swap:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 12 171 ki31 0K 192K CPU0 0 313.6H 1052.25% [idle]
12 root 62 -48 - 0K 992K WAIT 0 47.5H 163.87% [intr]
0 root 43 -68 0 0K 672K - 10 969:11 7.81% [kernel]
671 root 1 46 0 305M 294M select 8 35:34 4.20% /usr/local/sbin/bgpd -d -A 127.0.0.1
661 root 1 44 0 156M 144M select 7 14:08 0.10% /usr/local/sbin/zebra -d -A 127.0.0.1
...How I can configure the binding queues ix0 only CPU1 cores, and queues ix1 only CPU2 cores (eg reducing the number of queues to 6 on each interface)?