ZFS Best ashift for SAMSUNG MZVLB512HAJQ (PM981)

patpro · May 9, 2020

Hello,

I'm running a server with 4xSAS raidz plus 1xNVMe SSD (SAMSUNG MZVLB512HAJQ). The NVMe drive is split in 3 parts:

Code:

# zpool status
  pool: nvme
 state: ONLINE
  scan: none requested
config:

    NAME            STATE     READ WRITE CKSUM
    nvme            ONLINE       0     0     0
      gpt/nvmedata  ONLINE       0     0     0

errors: No known data errors

  pool: sas
 state: ONLINE
  scan: none requested
config:

    NAME             STATE     READ WRITE CKSUM
    sas              ONLINE       0     0     0
      raidz1-0       ONLINE       0     0     0
        da0p3        ONLINE       0     0     0
        da1p3        ONLINE       0     0     0
        da2p3        ONLINE       0     0     0
        da3p3        ONLINE       0     0     0
    logs
      gpt/nvmelog    ONLINE       0     0     0
    cache
      gpt/nvmecache  ONLINE       0     0     0

Every vdev uses ashift=12 but I've read in different places that SAMSUNG SSD like the PM981 could benefit from ashift=13.

Is it possible to change ashift for vdevs gpt/nvmedata, gpt/nvmelog, gpt/nvmecache (destroy and recreate) and still be able to add gpt/nvmelog & gpt/nvmecache back to my "sas" pool? (it'll end up mixing ashift=12 for SAS HDD and ashift=13 for NVMe backed vdevs).

I'm a bit worried that keeping ashift=12 could yield to write amplification on the NVMe drive, degrading performance where I need it most.

Phishfry · May 9, 2020

I was wondering the same about the PM983 drives I have.
ashift=12 or ashift=13
I could not find anything solid about the actual sector sizes my drives use after much research.
Something tells me they use 4K sectors but I couldn't find anything solid.
diskinfo shows 512K sectors which adds to the confusion.

D-FENS · May 9, 2020

Use the diskinfo command to see what is the physical sector size:
diskinfo gpt/nvmedata
Then you could use this value to determine which ashift is best to use.
I don't know if it is possible to change the ashift of an existing pool.

patpro · May 10, 2020

Hello, Samsung seems to have a history of not publishing internal details of its SSDs, and there can be a big difference between what's advertised by the SSD and what's actually going on on the NAND. Don't trust diskinfo about physical sector size on modern SSDs.

Phishfry · May 11, 2020

SmartMonTools shows both 512 and 4096 LBA sectors:

Code:

root@X10DRX:~ # smartctl -a /dev/nvme0
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-RELEASE amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-000AZ
Serial Number:                      S3VKNE0KA06251
Firmware Version:                   EDA53W0Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Sun May 10 19:17:14 2020 EDT
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
0 +     512       0         0
1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        42 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    1,487,293 [761 GB]
Data Units Written:                 5,855,760 [2.99 TB]
Host Read Commands:                 11,784,037
Host Write Commands:                34,712,775
Controller Busy Time:               59
Power Cycles:                       223
Power On Hours:                     325
Unsafe Shutdowns:                   126
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               42 Celsius
Temperature Sensor 2:               46 Celsius
Temperature Sensor 3:               52 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

Alain De Vos · May 11, 2020

What is advertised is frequently performant emulated.
I would choose the stripesize. If it's zero the sectorsize.

patpro · May 12, 2020

On mine I got only one LBA size:

Code:

$ sudo smartctl -a /dev/nvme0
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZVLB512HAJQ-00000
Serial Number:                      ****
Firmware Version:                   EXA7301Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 512,110,190,592 [512 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          512,110,190,592 [512 GB]
Namespace 1 Utilization:            137,216,131,072 [137 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 8891044992
Local Time is:                      Tue May 12 12:45:15 2020 CEST
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     81 Celsius
Critical Comp. Temp. Threshold:     82 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     7.02W       -        -    0  0  0  0        0       0
 1 +     6.30W       -        -    1  1  1  1        0       0
 2 +     3.50W       -        -    2  2  2  2        0       0
 3 -   0.0760W       -        -    3  3  3  3      210    1200
 4 -   0.0050W       -        -    4  4  4  4     2000    8000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        31 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    45,505 [23.2 GB]
Data Units Written:                 2,161,987 [1.10 TB]
Host Read Commands:                 2,335,064
Host Write Commands:                9,804,813
Controller Busy Time:               9
Power Cycles:                       32
Power On Hours:                     113
Unsafe Shutdowns:                   8
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               31 Celsius
Temperature Sensor 2:               43 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

And diskinfo won't provide any more details:

Code:

$ sudo diskinfo -v /dev/nvd0
/dev/nvd0
    512             # sectorsize
    512110190592    # mediasize in bytes (477G)
    1000215216      # mediasize in sectors
    0               # stripesize
    0               # stripeoffset
    SAMSUNG MZVLB512HAJQ-00000    # Disk descr.
    **************    # Disk ident.
    Yes             # TRIM/UNMAP support
    0               # Rotation rate in RPM

I'll stick with ashit=12.

Thank you all.

Alain De Vos · May 12, 2020

I found this,
nvmecontrol format [-f fmt] [-m mset] [-o pi] [-l pil] [-E] [-C] <device id> <namespace id>
[format
Format either specified namespace, or all namespaces of specified con-
troller, using specified parameters: fmt LBA Format]

patpro · May 13, 2020

Thanks, I'll probably stay way from any modifications though

Phishfry · May 13, 2020

patpro
Interesting to see the differences between PM981 and PM983.
The one thing that caught my eye was power states.
The M.2 drives have power states but only LBA 512 sectors and the U.2 drive offers either 512 or 4096 sector size but one power state.

I have to wonder how 4096 sectors would benchmark. Higher or lower.
I recently got some 1.92TB drives PM983 and have to take a peek at the settings.
They seem to benchmark considerably lower than 960GB drives.
2GB/sec with the 960Gb drives but 1.5GB/sec with the larger 1.92TB drives.

patpro · May 13, 2020

I've found this:
M.2 NVMe PCIe SSD specification (PM981) https://gzhls.at/blob/ldb/4/b/7/f/63615517d004b8ee7452f4f76469c7129df9.pdf
2.5"NVMe PCIe SSD Specification (PM983) https://gzhls.at/blob/ldb/2/b/4/9/d8f8e423e0134a3f116548621be52b49a323.pdf

Phishfry · Aug 26, 2021

Well any idea which ashift I should use? 12 or 13?
I am finally bringing up my 4 drive ZFS Striped Mirrors of 1.92 PM983 drives.
Creating that pool for life. Wanna get it right.

One of my 4 disks. This one has a newer firmware version and is faster than the rest.

Code:

root@X10DRX:~ # smartctl -a /dev/nvme2
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-RELEASE-p7 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB1T9HAJR-00007
Serial Number:                      S439NC0R306424
Firmware Version:                   EDA5402Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 1,920,383,410,176 [1.92 TB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,920,383,410,176 [1.92 TB]
Namespace 1 Utilization:            6,542,770,176 [6.54 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Thu Aug 26 01:53:12 2021 EDT
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        36 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    6,275 [3.21 GB]
Data Units Written:                 14,125 [7.23 GB]
Host Read Commands:                 87,582
Host Write Commands:                383,958
Controller Busy Time:               0
Power Cycles:                       135
Power On Hours:                     78
Unsafe Shutdowns:                   91
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               36 Celsius
Temperature Sensor 2:               41 Celsius
Temperature Sensor 3:               49 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

Still not sure how the get it to 4K sectors. 512 looks to be the default.

sko · Aug 26, 2021

Almost all disks nowadays should have 4k sectors; the very few ones that actually have 512byte sectors are usually marketed as "512n" and with a hefty premium on their price as they usually only make sense in database applications.
The (stupid) lying in the firmware about 512byte sectors we still see today was/is just a workaround for one very special and backwards operating system that also completely barfs when you install it in a VM on a volume with 4k sector size... (Yes, from the same family of OS we have to thank for still having to use a filesystem from the 80s on EFI partitions...)

Phishfry said:
Well any idea which ashift I should use? 12 or 13?

Just look at what would happen with 512 vs 4k filesystem sectors:
Using 512byte requests on a 4k disk increases the overhead by a factor of 8. You need 8 requests from the filesystem to get that full sector the drive reads in completely anyways.
OTOH using 4k filesystem sectors on a 512n disk would only mean that the drive firmware has to translate that one request into 8 distinct requests to the storage. You needed to access all of those 512byte sectors anyway, so the performance impact is miniscule if there even is any.
So it is always the safer option to go for 4k sectors on the filesystem side, especially if your pools are usually "here to stay". I've used ashift=13 during pool creation for several years now and even pools that have migrated from spinning rust to SSD and now to NVMe I've never seen any performance issues.

ralphbsz · Aug 26, 2021

Except that the disk we're talking about here is not spinning rust, but flash cells. It has a real block size, which is likely to be much larger than either 512 or 4K. There are interestingly complex performance implications of adjust the block size that the file system uses (ashift for ZFS), and bigger is not alway better. The biggest problem is that a larger ashift value may waste space. In my personal opinion, that is much less relevant than people typically think (since most space in a file system is typically used by a small fraction of large files), but depending on the usage and workload, the correct answer can matter significantly.

Alain De Vos · Aug 26, 2021

Is there a measurable performance improvement using ashift=13 vs ashift=12. Personally i don't think so.

sko · Aug 26, 2021

ralphbsz said:
Except that the disk we're talking about here is not spinning rust, but flash cells. It has a real block size, which is likely to be much larger than either 512 or 4K.

True, I really overlooked that when posting and my habit of using ashift=13 is still rooted in the use of spinning disks...
It would be interesting to see some benchmarks purely on fast NVMe drives to see if there really is any difference in 512 vs 4k "software blocksize"... OTOH - with current speeds of NVMe disks and especially when combining them into multiple mirror-vdevs, I suspect the fluctuations of the benchmarking tools/mechanisms are bigger than the actual impact on bandwidth and/or IOPS of using different blocksizes...

OTOH - for USB flash drives using a blocksize of 1M e.g. when writing images via dd increases speeds _drastically_ (but I never questioned if this might be caused by the USB protocol or the actual flash storage...)

Phishfry · Aug 26, 2021

Alain De Vos said:
nvmecontrol format

This looks to be the right path. Anyone actually use this?
I would assume it is not a physical format but firmware nudge?
Would I format the drive before setting up zfs?

I too would like to compare 512K versus 4K on a striped mirrored vdev.
With my speed back to expectation I am unsure I want to switch to 4K.
I have bricked enough gear to know I am not sure I want to mess with my precious 4 drives.
Might be a good experiment on a single 960GB PM983 drive. I have some of those that could be expendable.

Phishfry · Aug 26, 2021

sko said:
4k "software blocksize"

Interesting you quote that.
So from my reading some flash drives use pages instead of blocks and blocks are just translated.

Coding for SSDs – Part 3: Pages, Blocks, and the Flash Translation Layer | Code Capsule

This is Part 3 over 6 of "Coding for SSDs", covering Sections 3 and 4. For other parts and sections, you can refer to the Table to Contents. This is a series of articles that I wrote to share what I learned while documenting myself on SSDs, and on how to make code perform well

codecapsule.com

So I am sure NVMe have some flash translation. Do you think they use pages too?

Deeper Question. Why do we need software trim when the drives handle garbage collection at the controller level?

Phishfry · Aug 27, 2021

To throw some more information out there I have a Dell branded PCIe AIC version of PM1725.
This card has 4 different LBA modes possible.

Code:

moot@X10DRX:~ # smartctl -a /dev/nvme5
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-RELEASE-p7 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Dell Express Flash PM1725a 1.6TB AIC
Serial Number:                      S39WNA0J701025
Firmware Version:                   1.0.3
PCI Vendor ID:                      0x144d
PCI Vendor Subsystem ID:            0x1028
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 1,600,321,314,816 [1.60 TB]
Unallocated NVM Capacity:           0
Controller ID:                      33
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,600,321,314,816 [1.60 TB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 e771000405
Local Time is:                      Fri Aug 27 02:09:01 2021 EDT
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x0006):   Format Frmw_DL
Optional NVM Commands (0x0016):     Wr_Unc DS_Mngmt Sav/Sel_Feat
Maximum Data Transfer Size:         32 Pages
Warning  Comp. Temp. Threshold:     70 Celsius
Critical Comp. Temp. Threshold:     77 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    25.00W       -        -    0  0  0  0      100     100

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         1
 1 -     512       8         3
 2 -    4096       0         0
 3 -    4096       8         2

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        27 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    25,628,438 [13.1 TB]
Data Units Written:                 134,267,312 [68.7 TB]
Host Read Commands:                 206,926,593
Host Write Commands:                24,355,950,742
Controller Busy Time:               4,657
Power Cycles:                       55
Power On Hours:                     17,366
Unsafe Shutdowns:                   28
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               27 Celsius
Temperature Sensor 2:               23 Celsius
Temperature Sensor 3:               24 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

Benchmark:

Code:

loot@X10DRX:~ # diskinfo -wS /dev/nvd5
/dev/nvd5
    512             # sectorsize
    1600321314816    # mediasize in bytes (1.5T)
    3125627568      # mediasize in sectors
    0               # stripesize
    0               # stripeoffset
    Dell Express Flash PM1725a 1.6TB AIC    # Disk descr.
    S39WNA0J701025    # Disk ident.
    Yes             # TRIM/UNMAP support
    0               # Rotation rate in RPM

Synchronous random writes:
     0.5 kbytes:     19.9 usec/IO =     24.6 Mbytes/s
       1 kbytes:     19.9 usec/IO =     49.1 Mbytes/s
       2 kbytes:     19.9 usec/IO =     98.2 Mbytes/s
       4 kbytes:     19.6 usec/IO =    199.1 Mbytes/s
       8 kbytes:     21.9 usec/IO =    356.6 Mbytes/s
      16 kbytes:     25.5 usec/IO =    613.4 Mbytes/s
      32 kbytes:     33.0 usec/IO =    945.9 Mbytes/s
      64 kbytes:     44.4 usec/IO =   1409.2 Mbytes/s
     128 kbytes:     72.9 usec/IO =   1713.5 Mbytes/s
     256 kbytes:    136.0 usec/IO =   1838.3 Mbytes/s
     512 kbytes:    272.4 usec/IO =   1835.8 Mbytes/s
    1024 kbytes:    544.1 usec/IO =   1838.0 Mbytes/s
    2048 kbytes:   1071.1 usec/IO =   1867.2 Mbytes/s
    4096 kbytes:   2168.6 usec/IO =   1844.5 Mbytes/s
    8192 kbytes:   4309.2 usec/IO =   1856.5 Mbytes/s

ralphbsz · Aug 27, 2021

Phishfry said:
So I am sure NVMe have some flash translation. Do you think they use pages too?

The flash chip hardware has pages, so NVMe has to. In reality, NVMe is mostly just a new interface to the same old SSD architecture.

Deeper Question. Why do we need software trim when the drives handle garbage collection at the controller level?

Because garbage collection is much easier to do with trim. First, if there is no trim, the drive thinks that all data that was ever written is still valid, and needs to copy it around when doing garbage collection. With trim, it can skip moving things that are already deleted. Second, with trim the drive has much more free space available, making the number of moves less. You know that game with 16 squares, into which you can shift 15 little tiles with numbers? Imagine how easy that game would be if you only had to move 6 or 8 tiles, and all other spaces were free.

sko · Aug 27, 2021

ralphbsz said:
Second, with trim the drive has much more free space available, making the number of moves less. You know that game with 16 squares, into which you can shift 15 little tiles with numbers? Imagine how easy that game would be if you only had to move 6 or 8 tiles, and all other spaces were free.

I always had the impression random access is not a problem with flash storage? I.e. there's nothing that needs to move around (=a mechanical head) and software/firmware doesn't care if it asks for 20 succesive addresses or 20 completely random addresses...
So why is the data moved around anyways instead of just writing new data where there is still a free page?

Edit: I should have started reading that article linked by Phishfry before writing that answer. ?
Thanks for that link, this should definately clarify some (mis)conceptions I've had about flash storage (and opens up a whole new can of worms for me).

VladiBG · Aug 27, 2021

sko said:
So why is the data moved around anyways instead of just writing new data where there is still a free page?

When you have static data for example some big UHD or 4k movie file sitting on the same cells and other cells that are constantly changed by log files for example the cells that holding log files will wear off and the cells that holds the static data (the movie file) will not wear. That's why the data is moved around by the firmware on the background to reduce the wear of the same cells.

Phishfry · Aug 29, 2021

I figured out what to do to switch these PM983 to 4K sectors thanks to Alain De Vos and his advice.
I used the value from smartctl for the LBA mode. --lbaf=1
Smartctl shows the new default with a plus sign. It was quick but I did have to reset the drive.

Code:

~ # nvmecontrol format --lbaf=1 nvme1ns1
~ # smartctl -a /dev/nvme1
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-RELEASE-p7 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB1T9HAJR-00007
Serial Number:                      S439NA0M706128
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 1,920,383,410,176 [1.92 TB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,920,383,410,176 [1.92 TB]
Namespace 1 Utilization:            43,859,578,880 [43.8 GB]
Namespace 1 Formatted LBA Size:     4096
Local Time is:                      Sun Aug 29 00:39:07 2021 EDT
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         0
 1 +    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        37 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    28,180 [14.4 GB]
Data Units Written:                 136,004 [69.6 GB]
Host Read Commands:                 1,130,306
Host Write Commands:                1,587,350
Controller Busy Time:               0
Power Cycles:                       170
Power On Hours:                     136
Unsafe Shutdowns:                   103
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               37 Celsius
Temperature Sensor 2:               42 Celsius
Temperature Sensor 3:               48 Celsius

I also found a native command for benchmarking NVMe.
nvmecontrol perftest -n 32 -o read -s 512 -t 30 nvme0ns1

Phishfry · Aug 29, 2021

OK bad news. No increase in speed at any level.

Code:

root@X10DRX:~ # diskinfo -wS /dev/nvd1
/dev/nvd1
    4096            # sectorsize
    1920383410176    # mediasize in bytes (1.7T)
    468843606       # mediasize in sectors
    0               # stripesize
    0               # stripeoffset
    SAMSUNG MZQLB1T9HAJR-00007    # Disk descr.
    S439NA0M706128    # Disk ident.
    Yes             # TRIM/UNMAP support
    0               # Rotation rate in RPM

Synchronous random writes:
       4 kbytes:     17.6 usec/IO =    222.2 Mbytes/s
       8 kbytes:     19.3 usec/IO =    405.6 Mbytes/s
      16 kbytes:     23.6 usec/IO =    661.4 Mbytes/s
      32 kbytes:     29.7 usec/IO =   1050.8 Mbytes/s
      64 kbytes:     42.4 usec/IO =   1473.2 Mbytes/s
     128 kbytes:     68.6 usec/IO =   1823.1 Mbytes/s
     256 kbytes:    118.8 usec/IO =   2104.9 Mbytes/s
     512 kbytes:    235.2 usec/IO =   2125.7 Mbytes/s
    1024 kbytes:    468.8 usec/IO =   2133.2 Mbytes/s
    2048 kbytes:    938.9 usec/IO =   2130.2 Mbytes/s
    4096 kbytes:   1870.4 usec/IO =   2138.6 Mbytes/s
    8192 kbytes:   3758.2 usec/IO =   2128.7 Mbytes/s

Here is similar 512K sector drive. Do note it uses smaller blocksize for test.

Code:

root@X10DRX:~ # diskinfo -wS /dev/nvd2
/dev/nvd2
    512             # sectorsize
    1920383410176    # mediasize in bytes (1.7T)
    3750748848      # mediasize in sectors
    0               # stripesize
    0               # stripeoffset
    SAMSUNG MZQLB1T9HAJR-00007    # Disk descr.
    S439NC0R306424    # Disk ident.
    Yes             # TRIM/UNMAP support
    0               # Rotation rate in RPM

Synchronous random writes:
     0.5 kbytes:     16.8 usec/IO =     29.1 Mbytes/s
       1 kbytes:     17.6 usec/IO =     55.5 Mbytes/s
       2 kbytes:     18.4 usec/IO =    106.3 Mbytes/s
       4 kbytes:     17.5 usec/IO =    223.3 Mbytes/s
       8 kbytes:     19.1 usec/IO =    409.5 Mbytes/s
      16 kbytes:     23.8 usec/IO =    657.9 Mbytes/s
      32 kbytes:     29.7 usec/IO =   1052.2 Mbytes/s
      64 kbytes:     42.5 usec/IO =   1471.1 Mbytes/s
     128 kbytes:     68.1 usec/IO =   1834.2 Mbytes/s
     256 kbytes:    130.2 usec/IO =   1920.9 Mbytes/s
     512 kbytes:    252.6 usec/IO =   1979.4 Mbytes/s
    1024 kbytes:    497.4 usec/IO =   2010.6 Mbytes/s
    2048 kbytes:    958.2 usec/IO =   2087.2 Mbytes/s
    4096 kbytes:   1861.7 usec/IO =   2148.6 Mbytes/s
    8192 kbytes:   3718.1 usec/IO =   2151.6 Mbytes/s

VladiBG · Aug 29, 2021

Compare your results with the product specification provided by Samsung.