Ok, so here's the update. But before that, I just wanted to thank everyone how responded to this thread. This very much ignorant programmer is very thankful to you all.
As suggested by
Phishfry and @cracuer@ , we're opting for U.2 drives. However, my boss is still raising an eyebrow for their higher cost. I do have some questions, that maybe somebody can help me with:
1. Are Intel Optane U.2 drives any good? They seem to be discontinued, but are at a lower price than what was my first choice (the suggested Samsung 9A3).
2. The Supermicro board has 4 PCI-E 4.0 NVMe x4 internal ports, so 4 U.2 drives will go there, in a zpool.
3. I still have the Areca 1886 RAID Card. I guess some time in the future, I could use that for attaching some decomissioned regular hard drives if needed for locally accesing a replica of the PostgreSQL we use for development.
However, there's something I'm confused about: That Areca card requires a PCIe x8 slot. I just spent some time reading its docs, and Areca states that it's a trimode card, with 2 ports. Docs state that I can attach 4 U.2 drives to the card, instead of the 16 hard drives. At 4 lanes per U.2 drive, that's 16. How can those U.2 drives perform at NVMe speed if they're connected via an x8 slot? When I asked Areca tech support, this is what I got:
"No onboard PCIe switch, the onboard processor have 16 lanes to devices, 8 lanes to PCIe slot. So each NVMe drive on 1686-4NOD can work with x4 speed."
Thanks for reading!