Dumb Question: But are NVME Drives Supported Well?

Are nvme drives supported well for boot drives? I was gonna buy a Thinkcentre M720 Tiny for pfsense and I wanted to know if there were gonna be issues with nvme drives. A user on reddit said they had issues with a nvme drive not showing up in the install but showed up in the bios and getting errors or something. But i made another post and there were some people saying they had no issues. Would the hardware i put in the photo be good for pfsense?



6815
 
In any case, I have several on different systems (running 12.0 or CURRENT), and seeing no issues with NVMe at all. E.g. one on NUC7i7BNH:
Code:
...
ACPI APIC Table: <INTEL  NUC7i7BN>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
...
nvme0: <Generic NVMe Device> mem 0xdc000000-0xdc003fff at device 0.0 on pci4
...
nvd0: <INTEL SSDPEKKW128G8> NVMe namespace
nvd0: 122104MB (250069680 512 byte sectors)
...
 
In any case, I have several on different systems (running 12.0 or CURRENT), and seeing no issues with NVMe at all. E.g. one on NUC7i7BNH:
Code:
...
ACPI APIC Table: <INTEL  NUC7i7BN>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
...
nvme0: <Generic NVMe Device> mem 0xdc000000-0xdc003fff at device 0.0 on pci4
...
nvd0: <INTEL SSDPEKKW128G8> NVMe namespace
nvd0: 122104MB (250069680 512 byte sectors)
...


Gotcha, looks good. Idk what that guy was saying since i asked on reddit and some other people were using nvme drives no problem. I guess FreeBSD does have good kernal support for nvme drives
 
I hope you learned your lesson. Never, EVER trust ANYTHING you EVER read on reddit. 80% of all users there are kids under 18 with no job experience whatsoever. While some subs have some valuable members in them, you have to wade through 80% of bogus support (BS for short) to find out who they are. It's never worth the effort.


Yea lmaoo. This is my first time ever using pfsense or freebsd and when he told me he was having issues in the past with nvme ssds i listened since i didnt want to have to be stuck with hardware i couldn't use. Then i made another post and people were like "I use nvme, no issues" 🙄

Its also possible he was teying to scare me off from that deal i got on the lenovo M720 since for pfsense that thing is basically overkill, idk

But thanks for reassuring me
 
Gotcha, looks good. Idk what that guy was saying since i asked on reddit and some other people were using nvme drives no problem. I guess FreeBSD does have good kernal support for nvme drives
There are NVMe drives and there are NVMe drives.:p

Seriously, every NVMe drive I have tried, except one, works perfectly on FreeBSD out-of-the-box. The remaining one, an OEM Samsung something-or-other, did not work until the flaw (it not being quite compliant with the specification) was finally identified and worked around (the fix has probably not landed outside of HEAD yet).

However, in the bug report I mentioned, a bunch of people reported unrelated problems with suspend / resume on various NVMe drives on various notebooks. Suspend / resume is unfortunately one of the well-specified parts of the PC where manufacturers often ignore the specs and do their own thing, because they expect their users to be running one of their supported operating systems (Windows, and if you're lucky, some distro of Linux in a specific version which is no longer current). Getting the information needed to deal with these manufacturer-induced oddities is often impossible. So, when testing to see if any piece of hardware (not just NVMe drives) works on a notebook, regardless of whether you're running FreeBSD, Linux, or something else, you should always check the various types of suspend / resume / hibernate to make sure things are working.
 
There are NVMe drives and there are NVMe drives.:p

Seriously, every NVMe drive I have tried, except one, works perfectly on FreeBSD out-of-the-box. The remaining one, an OEM Samsung something-or-other, did not work until the flaw (it not being quite compliant with the specification) was finally identified and worked around (the fix has probably not landed outside of HEAD yet).

However, in the bug report I mentioned, a bunch of people reported unrelated problems with suspend / resume on various NVMe drives on various notebooks. Suspend / resume is unfortunately one of the well-specified parts of the PC where manufacturers often ignore the specs and do their own thing, because they expect their users to be running one of their supported operating systems (Windows, and if you're lucky, some distro of Linux in a specific version which is no longer current). Getting the information needed to deal with these manufacturer-induced oddities is often impossible. So, when testing to see if any piece of hardware (not just NVMe drives) works on a notebook, regardless of whether you're running FreeBSD, Linux, or something else, you should always check the various types of suspend / resume / hibernate to make sure things are working.

Since the nvme drive will go in a desktop that will basically be on 24/7 suspend/resume wont be an issue thankfully. I was planning on using the drive for a pfsense build since the nvme drive is included already. This thing would be a beast for my home fiber. Like the other guy said, the reddit dude was most likely misinformed or has some weird drive in it. Usually lenovo uses Hynix drives for their lower end storage capacities.
 

Attachments

  • 7096A0B9-906B-47C1-BD43-3751FA4904EC.jpeg
    7096A0B9-906B-47C1-BD43-3751FA4904EC.jpeg
    308.2 KB · Views: 305
I'm using a NVMe boot drive with an Intel 9900k CPU and it's been faultless. What I did discover on my way to that particular installation is that NVMe booting is not fully supported on a Sandy Bridge CPU. (Thread here.)
 
I'm using a NVMe boot drive with an Intel 9900k CPU and it's been faultless. What I did discover on my way to that particular installation is that NVMe booting is not fully supported on a Sandy Bridge CPU. (Thread here.)


Thats good to hear. Id assume newer intel cpus like the 8th and 9th gen to support nvme no problemo, but i thought Sandy Bridge didn't support nvme drives natively which is the reason why for the incompatibility?
 
but i thought Sandy Bridge didn't support nvme drives natively which is the reason why for the incompatibility?
Well LGA2011 SandyBridge did have PCIe 3.0. I know the E5-26xx series does.
But most consumer SandyBridge boards didn't. Those that did needed a IvyBridge BIOS flash for PCIe 3.0.
Even with IvyBridge, the EFI module needed in EFI BIOS to boot from NVMe is not present in most boards.
BIOS Hackers slip it in no problem though. So more like manufacturers did not want to add features to old board.
For that reason I say Haswell is probably the first gen boards that mostly supported booting from NVMe.

I would also like to say that M.2 format drives are not optimal.
2.5" U.2 NVMe drives like Samsung PM-983 are better suited to regular PC usage.
A simple paddle board PCIe controller and U.2 cable does make them usable in many computers.
M.2 being such a small factor makes it harder to disperse heat. NVMe throttle down to maintain a thermal envelope.
Most M.2 are best off for notebooks. In a PC a cooling fan is recommended for best NVMe performance.
Here is what I own and work with FreeBSD:
Samsung PM-953 (110mm long M.2) I have a pair of these on SuperMicro AOC-SLG3-2M2 on SuperMicro X10DRI serving VM's.
Samsung PM-983 (2.5" U.2) I have 4 of these in various computers. My quickest drives.
Toshiba XG3 (80mm M.2) These are not quick compared to my newer Samsung. I use them with M.2 to PCIe adapter cards.
 
NVMe works fine here:
Code:
root@kg-core2# nvmecontrol devlist
 nvme0: INTEL SSDPEKNW010T8
    nvme0ns1 (976762MB)
on FreeBSD version
Code:
root@kg-core2# freebsd-version -ku
11.3-RELEASE-p1
11.3-RELEASE-p1
HTH
 
Here is what I am calling a NVMe paddle card useful for 2.5" NVMe drives.

Here are the NVMe cables I use. I like Amphenol and pay extra for the name.
 
Here is a good example of what I am talking about.
Two Sandy Bridge chips. One with PCIe 2 and one with PCIe 3 bus.

So all LGA2011 have the ability to run NVMe at full speed. All SuperMicro LGA2011 boards that I tested will boot it in EFI mode.

Now stick that consumer Sandy Bridge LGA1155 into an IvyCreek board and it will only have a PCIe 2.0 bus.
The CPU dictates the PCIe bus speed. Memory speed too as controller is on chip.
 
Yeah the U.2 drive configuration is really nice. I'd like to use that as well, unfortunately consumer interests drive makers to M.2. U.2 is better, but it's considered enterprise with lower availability and higher pricing.

I've been using a removable system drive in my personal desktop machines for a long time. For test installs it's more simple for me to use a "scratch-pad" disk rather than a virtual machine. It's just easier for me and I prefer doing it that way.

U.2 would be nice for me since I like to use a removable system drive, however, there's not many U.2 drive makers other than Intel and prices reflect that. Selection is really thin for drive racks and no consumer motherboard makers provide the connector for one. You have to use an M.2 to SFF-8639 adapter. So the market is pretty much forcing me into using M.2 drives.
 
I did have a NVME drive that wasn't supported very well - a Samsung PM951. It basically wouldn't even boot on RELEASE but it worked just fine on STABLE. But I think thats the exception rather than the rule.
 
There is a probe of Samsung PM951 on 13.0-CURRENT - everything looks fine now.

The list of tested controllers from the database (with at least one NVMe drive installed and reporting SMARTs as operability test):

VendorBusTypeName
ADATA TechnologyPCIstorage/nvmeXPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive »
IntelPCIstorage/nvmeSSD 660P Series »
IntelPCIstorage/nvmeSSD Pro 7600p/760p/E 6100p Series / Corporation SSD Pro 7600p/760p/E 6100p Series [NVM Express] »
Phison ElectronicsPCIstorage/nvmeE12 NVMe Controller »
Samsung ElectronicsPCIstorage/nvmeNVMe SSD Controller SM951/PM951 / Surface NVMe Controller »
Samsung ElectronicsPCIstorage/nvmeNVMe SSD Controller SM961/PM961 / Surface NVMe Controller »
Samsung ElectronicsPCIstorage/nvmeNVMe SSD Controller SM981/PM981/PM983 / NVMe Controller »
SandiskPCIstorage/nvmeWD Black 2018/PC SN520 NVMe SSD »
SandiskPCIstorage/nvmeWD Black 2018/PC SN720 NVMe SSD »
ToshibaPCIstorage/nvmeBG3 NVMe SSD Controller / Surface NVM Express Controller »
ToshibaPCIstorage/nvmeXG4 NVMe SSD Controller »
 
running a "PNY CS3030 1TB SSD" and a "GIGABYTE GP-GSM2NE3100TNTD" on my desktop, some others on 3 notebooks I currently have no access ... all with 12.1, no problems so far
 
I've got a Samsung 970 PRO (1 TB), plugged into an ASUS mainboard (“ROG Crosshair VII Hero”) with AMD Ryzen 2700. I've never had a storage drive that came close to this in speed. Love it.

BTW, I'm using FreeBSD stable/12 r362094 (2020-06-12) with the setting hw.nvme.use_nvd=0 so the kernel uses the nda(4) driver instead of the legacy nvd(4) driver that is still the default in FreeBSD 12.
 
Last edited:
BTW, I'm using FreeBSD stable/12 r362094 (2020-06-12) with the setting hw.nvme.use_nvd=0 so the kernel uses the nda(4) driver instead of the default nvd(4) driver that is still the default in FreeBSD 12.
Interesting to hear, I have not read anything about that. May I ask what's the benefit of nda(4) over nvd(4), or whats different?
 
Interesting to hear, I have not read anything about that. May I ask what's the benefit of nda(4) over nvd(4), or whats different?
Two things:
  • FreeBSD 13-current is moving towards making nda(4) the default, and nvd(4) is probably going away at some point in the future. So now is a good time to start using nda(4) on 12-stable, too.
  • As Phishfry mentioned, nda(4) uses the standard CAM infrastructure, as do other storage drivers like ada(4) and da(4), so you can use camcontrol(8) and other standard tools. Accessing SMART data (using the sysutils/smartmontools port) works, too, of course. A small correction, though: You can still use nvmecontrol(8) because it is used for management of the NVMe controller and namespaces, while camcontrol(8) is used for management of the actual storage device.
 
Back
Top