pciconf, pcie version

[braddeicide]

My motherboard has PCIe 2.0, and the device is PCIe 3.0, I've got the correct x4 lanes, but the speed appears to be PCIe 1 out of the devices ability to do 8.0. I've got other devices in the system running at 5.0, I'd expect to see 5.0(8.0).

nvme0@pci0:3:0:0:       class=0x010802 card=0xa801144d chip=0xa802144d rev=0x01 hdr=0x00
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 8 messages, 64 bit
    cap 10[70] = PCI-Express 2 endpoint max data 128(128) FLR NS
[B]                 link x4(x4) speed 2.5(8.0) ASPM disabled(L1)[/B]
    cap 11[b0] = MSI-X supports 9 messages, enabled
                 Table in map 0x10[0x3000], PBA in map 0x10[0x2000]
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[148] = Serial 1 0000000000000000
    ecap 0004[158] = Power Budgeting 1
    ecap 0019[168] = PCIe Sec 1 lane errors 0
    ecap 0018[188] = LTR 1
    ecap 001e[190] = unknown 1

What can cause a single device to get a lower speed that others?

v. 1.x (2.5 GT/s):
v. 2.x (5 GT/s):
v. 3.x (8 GT/s):

 

[Phishfry]

Are you using a M.2 NVMe drive or is this a U.2 drive? Perhaps it is a PCIe card or PCIe to M.2 adapter card.
Motherboard model might help.

Show us actual throughtput numbers from FreeBSD with diskinfo -t for comparison.

Some older motherboards with PCIe 2.0 slots would not see the drive from my experiences.

 

[braddeicide]

It's a M.2 NVMe on a PCIe to M.2 Adapter card. The card has zero chips on it so it didn't seem like anything more than a physical plug adaptor. It claims v1,2,3 motherboard compatibility.

nvd0
        512             # sectorsize
        512110190592    # mediasize in bytes (477G)
        1000215216      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset

Seek times:
        Full stroke:      250 iter in   0.016037 sec =    0.064 msec
        Half stroke:      250 iter in   0.017652 sec =    0.071 msec
        Quarter stroke:   500 iter in   0.034617 sec =    0.069 msec
        Short forward:    400 iter in   0.028121 sec =    0.070 msec
        Short backward:   400 iter in   0.028440 sec =    0.071 msec
        Seq outer:       2048 iter in   0.036239 sec =    0.018 msec
        Seq inner:       2048 iter in   0.036883 sec =    0.018 msec
Transfer rates:
        outside:       102400 kbytes in   0.154463 sec =   662942 kbytes/sec
        middle:        102400 kbytes in   0.150354 sec =   681059 kbytes/sec
        inside:        102400 kbytes in   0.149984 sec =   682739 kbytes/sec

When I only had a 1x lane I was getting 195MB/s.

 

[Phishfry]

I bought one nvme and I have a thread here with my details.
https://forums.freebsd.org/threads/58170/

In a nutshell many motherboards have lots of slots that can only be run at a fraction of their true potential.
Alot of times PCIe slots are described in EE terms as Electrical Spec and also Physical Spec.
So for example my server board has an x16 slot and x8 slot. But the board can only supply 2each x8 lanes electrically.
You really see the effects with high end gamers and multiple video cards. Many times they have one card at x16 and 3 others at x8. Not by choice.
Intel has just not kept up in the way of PCIe lanes and board manufacturers just stuff slots on the boards with poor documentation. Many times with conflicting info in their own manuals.