Most of by previous thumb drives were either too slow or too small to warrant serious I/O benchmarks. However, I just acquired a few PNY 128 GB USB 3.1 memory sticks. They cost US$20 each (in Australia), and are regarded as being reasonably good performers, and I wanted to understand just how good.
I just ran benchmarks/bonnie++ across the fleet, and I threw in a variety of spinning disks and SSDs for comparison with the PNY memory sticks.
Here is a summary of the results:
The "per character" (stdio) tests were not done (they take an eternity, and are just a proxy for some type of CPU benchmark).
Media:
F/S: File system type in which test files were written. Linux systems use ext4 and xfs. FreeBSD systems use ufs and zfs.
The Block Outputs, Rewrites, and Inputs are megabytes or gigabytes per second.
Random Seeks are seeks per second.
%CP is the percentage of a single CPU used by running the test.
I have been trying to figure out what the "+++" fields mean, and I think I need to read the code. The easy way out is to assert a field overflow, but I'm somewhat uneasy about that.
My impressions when building systems with USB 3.1 thumb drives was that their performance was satisfactory for many tasks. The benchmark results show why. In comparing the Velociraptor spinning disks to the PNY, the PNY thumb drive is:
It's worth noting that the FreeBSD USB storage drivers use a fair bit less CPU capacity than their Linux counterparts.
The M.2 Samsung SSD is rated at PCIe Gen 3.0 x4. However it's connected through an x1 slot on the motherboard, which is a distinct limitation. NVMe should be able to do much better than shown above when configured favourably.
The ZFS tank is awesome!
There's a lot of uncontrolled variables in the above tests. Issues like duty cycle, latency, and redundancy may lead you away from relying on a USB stick. Additionally, not all USB 3.1 thumb drives perform well. But I think that a good USB 3.1 thumb drive would provide a viable option as system disk for a wide variety of applications with casual I/O needs, like media clients, routers, switches, and maybe even desktops.
I just ran benchmarks/bonnie++ across the fleet, and I threw in a variety of spinning disks and SSDs for comparison with the PNY memory sticks.
Here is a summary of the results:
Code:
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
/sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
Media Mem/File F/S
------------------
PNY 16/64G ext4 71.0m 15 51.1m 7 286m 18 1376 64
PNY 16/64G xfs 71.3m 10 52.4m 7 293m 18 1539 43
PNY 32/92G ufs 70.3m 3 56.2m 2 286m 6 1549 9
VRap 8/64G xfs 102m 12 54.8m 10 139m 26 170 6
MX500 32/128G xfs 253m 8 161m 7 507m 16 5392 121
NVMe 16/64G xfs 773m 47 366m 37 668m 49 3251 139
Tank 32/128G zfs 2.3g 85 1.8g 93 3.8g 99 +++++ +++The "per character" (stdio) tests were not done (they take an eternity, and are just a proxy for some type of CPU benchmark).
Media:
- PNY: PNY 128 GB USB 3.1 memory stick (in a USB 3.1 or 3.2 slot)
- VRap: WDC WD2500HHTZ-0 250GB SATA Velociraptor spinning disks (mirror pair)
- MX500: Crucial CT250MX500SSD1 250GB SATA SSD (mirror pair)
- NVMe: M.2 Samsung SSD 970 EVO Plus 500GB NVMe on PCIe x1 slot (mirror pair)
- Tank: zfs server tank, 4 x 3TB mirrors, striped (spinning SATA disks)
F/S: File system type in which test files were written. Linux systems use ext4 and xfs. FreeBSD systems use ufs and zfs.
The Block Outputs, Rewrites, and Inputs are megabytes or gigabytes per second.
Random Seeks are seeks per second.
%CP is the percentage of a single CPU used by running the test.
I have been trying to figure out what the "+++" fields mean, and I think I need to read the code. The easy way out is to assert a field overflow, but I'm somewhat uneasy about that.
My impressions when building systems with USB 3.1 thumb drives was that their performance was satisfactory for many tasks. The benchmark results show why. In comparing the Velociraptor spinning disks to the PNY, the PNY thumb drive is:
- 70% as fast as the Velociraptor for sequential output;
- has equivalent file re-write speeds;
- twice as fast as the Velociraptor for sequential input; and
- about nine times faster than the Velociraptor for random seeking.
It's worth noting that the FreeBSD USB storage drivers use a fair bit less CPU capacity than their Linux counterparts.
The M.2 Samsung SSD is rated at PCIe Gen 3.0 x4. However it's connected through an x1 slot on the motherboard, which is a distinct limitation. NVMe should be able to do much better than shown above when configured favourably.
The ZFS tank is awesome!
There's a lot of uncontrolled variables in the above tests. Issues like duty cycle, latency, and redundancy may lead you away from relying on a USB stick. Additionally, not all USB 3.1 thumb drives perform well. But I think that a good USB 3.1 thumb drive would provide a viable option as system disk for a wide variety of applications with casual I/O needs, like media clients, routers, switches, and maybe even desktops.