Hi all!
My question is addressed to experienced administrators like vermaden or Michael W. Lucas, so if it's not difficult, write to the point.
At the moment, we are evaluating the possibility of self building a storage system based on open source software. Our requirement requirements are a typical entry-level middle range of 200TB of usable capacity, which is expandable to 600 - 1000TB. 20% of storage volume - SSD, remain 80% - NL SAS 7.2 The main purpose of the storage system is a Postgresql database with replication, which does not have a high OLTP load, a WEB application server, file storage. There are few users - less than 100. Connection type - we are planning 10 Gbe either NFS or ISCSI. Ideally, I would like to get a failover configuration based on two servers in the form of a failover cluster connected via SAS to JBOD in which there are 2-port SAS like the TrueNAS® Enterprise.
We consider ZFS as a file system - since the pitfalls of BTRFS are well described in the Debian Wiki. The OS has not yet been decided, I myself have been using Debian Root ZFS for two years - this summer I worked remotely at the dacha, sometimes the lights were turned off 4 times a day - ZFS version 2.X.X proved to be worthy - there was no data loss. In Debian, the only concern is the short release life of 5 years. An attempt to install ZFS on Alma/Rocky 8 was unsuccessful. FreeBSD has ZFS out of the box - but not all hardware detects it. The question is still open.
Questions
1. Is it possible to assemble and configure the analogue of two controller TrueNAS® Enterprise via SAS multipath using some standard software from the FreeBSD distribution. Michael W. Lucas writes in Advanced ZFS Chapter 6: Advanced Hardware Sesutil(8) about multipath - but how do you make it so that if one node exits, the second node automatically continues and becomes active without losing data?
2. If there is no software for creating an analogue of TrueNAS® Enterprise on two nodes, then what options should be taken into consideration HAST (Network RAID1 of two identical servers) or Gluster (is it possible to build a cluster of only 2 servers)? What is the operating experience of HAST or Gluster
3. Since FreeBSD has a rolling release, did the server crash after the upgrade?
4. Question wermaden - you self built FreeBSD Enterprise 1 PB Storage. Experience was successful? Does everything work fine on the one node?
5. Planned RAID levels - SSD RAID1, NL 7.2 - RAID6 - what are the recommendations for operating experience for the optimal number of drives in the RAID 6 analog for ZFS? More than how many disks in the analog of RAID 6 does ZFS fall performance?
6. What free software would you recommend for backup and which backup model in terms of reliability 1) Backup to deduplicator? 2) Backup directly to tape with storage through SAS? 3) Backup via a backup server with a SAS streamer?
Best Regards, Vadim Maklakov.
My question is addressed to experienced administrators like vermaden or Michael W. Lucas, so if it's not difficult, write to the point.
At the moment, we are evaluating the possibility of self building a storage system based on open source software. Our requirement requirements are a typical entry-level middle range of 200TB of usable capacity, which is expandable to 600 - 1000TB. 20% of storage volume - SSD, remain 80% - NL SAS 7.2 The main purpose of the storage system is a Postgresql database with replication, which does not have a high OLTP load, a WEB application server, file storage. There are few users - less than 100. Connection type - we are planning 10 Gbe either NFS or ISCSI. Ideally, I would like to get a failover configuration based on two servers in the form of a failover cluster connected via SAS to JBOD in which there are 2-port SAS like the TrueNAS® Enterprise.
We consider ZFS as a file system - since the pitfalls of BTRFS are well described in the Debian Wiki. The OS has not yet been decided, I myself have been using Debian Root ZFS for two years - this summer I worked remotely at the dacha, sometimes the lights were turned off 4 times a day - ZFS version 2.X.X proved to be worthy - there was no data loss. In Debian, the only concern is the short release life of 5 years. An attempt to install ZFS on Alma/Rocky 8 was unsuccessful. FreeBSD has ZFS out of the box - but not all hardware detects it. The question is still open.
Questions
1. Is it possible to assemble and configure the analogue of two controller TrueNAS® Enterprise via SAS multipath using some standard software from the FreeBSD distribution. Michael W. Lucas writes in Advanced ZFS Chapter 6: Advanced Hardware Sesutil(8) about multipath - but how do you make it so that if one node exits, the second node automatically continues and becomes active without losing data?
2. If there is no software for creating an analogue of TrueNAS® Enterprise on two nodes, then what options should be taken into consideration HAST (Network RAID1 of two identical servers) or Gluster (is it possible to build a cluster of only 2 servers)? What is the operating experience of HAST or Gluster
3. Since FreeBSD has a rolling release, did the server crash after the upgrade?
4. Question wermaden - you self built FreeBSD Enterprise 1 PB Storage. Experience was successful? Does everything work fine on the one node?
5. Planned RAID levels - SSD RAID1, NL 7.2 - RAID6 - what are the recommendations for operating experience for the optimal number of drives in the RAID 6 analog for ZFS? More than how many disks in the analog of RAID 6 does ZFS fall performance?
6. What free software would you recommend for backup and which backup model in terms of reliability 1) Backup to deduplicator? 2) Backup directly to tape with storage through SAS? 3) Backup via a backup server with a SAS streamer?
Best Regards, Vadim Maklakov.