How to monitor PCI-E usage?

Hi,

I am currently trying to find out the limited factor on a Netmap/Chelsio system where we do not get high-enough performance. I would like to validate that we are not limited by PCI-E bandwidth. Is there any tool that would give the used PCI-E bandwidth? I have done that multiple times on Linux but can't find a tool to do that on FreeBSD.

Any idea?

Thanks!

Baptiste
 
Code:
% pkg info --list pciutils | grep bin
        /usr/local/bin/lspci
        /usr/local/bin/setpci
% pkg query %o pciutils
sysutils/pciutils
%

HTH

Plus the topic below, although it might not have the answer.

 
# lspci | grep Broadcom
0a:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2

# lspci -vv -s 0a:00 | grep LnkSta:
LnkSta: Speed 8GT/s, Width x8

# lspci -vv -s 0a:00 | grep LnkCap2
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-

I was unable to find where it specified the PCIe version
 
I was unable to find where it specified the PCIe version

Speed 8GT/s,
That would be your PCIe version.
From the Comparison Table
8GT=PCIe3

The Intel pmc package contains that.
You mean PCM package I believe..

sysutils/intel-pcm

 
It looked nice in the brochure....

Code:
xorg@E6420:~ % pcm-iio

 Intel(r) Performance Counter Monitor (2024-05-23 09:12:02 +0200 ID=e9a1f396)

 This utility measures IIO information


=====  Processor information  =====
Hybrid processor         : no
IBRS and IBPB supported  : no
STIBP supported          : no
Spec arch caps supported : no
Max CPUID level          : 13
CPU model number         : 42
cpuctl(4) not loaded.

Detected Intel(R) Core(TM) i3-2330M CPU @ 2.20GHz "Intel(r) microarchitecture codename Sandy Bridge" stepping 7
Failed to discover pci tree: unknown platform
 
I thought mabe root access needed to monitor pcie stats but it did not help.
In all fairness a laptop is not the best test with no PCIe slots...
Still I expected to see Networking via MiniPCIe slot wifi card... Only PCIe x1

Code:
root@E6420:~ # pcm-iio

 Intel(r) Performance Counter Monitor (2024-05-23 09:12:02 +0200 ID=e9a1f396)

 This utility measures IIO information


=====  Processor information  =====
Hybrid processor         : no
IBRS and IBPB supported  : no
STIBP supported          : no
Spec arch caps supported : no
Max CPUID level          : 13
CPU model number         : 42
Number of physical cores: 2
Number of logical cores: 4
Number of online logical cores: 4
Threads (logical cores) per physical core: 2
Num sockets: 1
Physical cores per socket: 2
Last level cache slices per socket: 2
Core PMU (perfmon) version: 3
Number of core PMU generic (programmable) counters: 4
Width of generic (programmable) counters: 48 bits
Number of core PMU fixed counters: 3
Width of fixed counters: 48 bits
Nominal core frequency: 2200000000 Hz
Package thermal spec power: 35 Watt; Package minimum power: 24 Watt; Package maximum power: 44 Watt;

Socket 0: 0 PCU units detected. 0 IIO units detected. 0 IRP units detected. 0 CHA/CBO units detected. 0 MDF units detected. 0 UBOX units detected. 0 CXL units detected. 0 PCIE_GEN5x16 units detected. 0 PCIE_GEN5x8 units detected.


Detected Intel(R) Core(TM) i3-2330M CPU @ 2.20GHz "Intel(r) microarchitecture codename Sandy Bridge" stepping 7 microcode level 0x29
Failed to discover pci tree: unknown platform
Cleaning up
 Zeroed PMU registers
 Zeroed uncore PMU registers
 
HTH

Plus the topic below, although it might not have the answer.
Unfortunately, lspci will only tell you how it's connected/configured. It won't tell you if you are close to reacht the bandwidth. If you have 8G and use 2G or if you use 7.9G is a different story and this is what I am looking for.
The Intel pmc package contains that. If you have an AMD CPU you are currently out of luck.
Unfortunately, this machine is on AMD :( But good to know that we have good support on Intel.
 
… Is there a way (a utility) to measure the volume of the data going through a specific PCI-Express slot? …

pcm-pcie, maybe?

Code:
root@mowa219-gjp4-zbook-freebsd:~ # pcm-pcie

 Intel(r) Performance Counter Monitor: PCIe Bandwidth Monitoring Utility
 This utility measures PCIe bandwidth in real-time

 PCIe event definitions (each event counts as a transfer):
   PCIe read events (PCI devices reading from memory - application writes to disk/network/PCIe device):
     PCIePRd   - PCIe UC read transfer (partial cache line)
     PCIeRdCur* - PCIe read current transfer (full cache line)
         On Haswell Server PCIeRdCur counts both full/partial cache lines
     RFO*      - Demand Data RFO
     CRd*      - Demand Code Read
     DRd       - Demand Data Read
     PCIeNSWr  - PCIe Non-snoop write transfer (partial cache line)
   PCIe write events (PCI devices writing to memory - application reads from disk/network/PCIe device):
     PCIeWiLF  - PCIe Write transfer (non-allocating) (full cache line)
     PCIeItoM  - PCIe Write transfer (allocating) (full cache line)
     PCIeNSWr  - PCIe Non-snoop write transfer (partial cache line)
     PCIeNSWrF - PCIe Non-snoop write transfer (full cache line)
     ItoM      - PCIe write full cache line
     RFO       - PCIe partial Write
   CPU MMIO events (CPU reading/writing to PCIe devices):
     UCRdF     - read from uncacheable memory, including MMIO
     WCiL      - streaming store (partial cache line), includes MOVDIRI

     WCiLF     - streaming store (full cache line), includes MOVDIR64

     PRd       - MMIO Read [Haswell Server only] (Partial Cache Line)
     WiL       - MMIO Write (Full/Partial)

 * - NOTE: Depending on the configuration of your BIOS, this tool may report '0' if the message
           has not been selected.


=====  Processor information  =====
Hybrid processor         : no
IBRS and IBPB supported  : yes
STIBP supported          : yes
Spec arch caps supported : no
Max CPUID level          : 13
CPU model number         : 60
Number of physical cores: 4
Number of logical cores: 8
Number of online logical cores: 8
Threads (logical cores) per physical core: 2
Num sockets: 1
Physical cores per socket: 4
Last level cache slices per socket: 4
Core PMU (perfmon) version: 3
Number of core PMU generic (programmable) counters: 4
Width of generic (programmable) counters: 48 bits
Number of core PMU fixed counters: 3
Width of fixed counters: 48 bits
Nominal core frequency: 2500000000 Hz
IBRS enabled in the kernel   : yes
STIBP enabled in the kernel  : yes
Package thermal spec power: 47 Watt; Package minimum power: 0 Watt; Package maximum power: 0 Watt;

Socket 0: 0 PCU units detected. 0 IIO units detected. 0 IRP units detected. 0 CHA/CBO units detected. 0 MDF units detected. 0 UBOX units detected. 0 CXL units detected. 0 PCIE_GEN5x16 units detected. 0 PCIE_GEN5x8 units detected.

For non-CSV mode delay < 1.0s does not make a lot of practical sense. Default delay 1s is used. Consider to use CSV mode for lower delay values
Update every 1 seconds

Detected Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz "Intel(r) microarchitecture codename Haswell" stepping 3 microcode level 0x28
Jaketown, Ivytown, Haswell, Broadwell-DE, Skylake, Icelake, Snowridge and Sapphirerapids Server CPU is required for this tool! Program aborted
Cleaning up
 Zeroed PMU registers
 Zeroed uncore PMU registers
root@mowa219-gjp4-zbook-freebsd:~ #

wichtounet sorry, not AMD.
 
SandyBridge was too old to work with this tool.

Jaketown, Ivytown, Haswell, Broadwell-DE, Skylake, Icelake, Snowridge and Sapphirerapids Server CPU is required for this tool!
Program aborted
 
Back
Top