BSD-based fw/router: ZFS on SSD RAID10 efficiency

Phishfry · May 8, 2023

I want to highly recommend security/tripwire for firewall usage.

It takes a bit to setup but once tuned it is the ultimate sentry.

Al Seitsinger · May 8, 2023

Phishfry said:
I want to highly recommend security/tripwire for firewall usage.

It takes a bit to setup but once tuned it is the ultimate sentry.

I would suggest making use of /usr/sbin/mtree instead, which comes with the base system. Manpage: mtree(8)

Phishfry · May 10, 2023

Phishfry said:
I disagree. FreeBSD base is around 3GB without source or ports tree.
4GB is sufficient.

Well I will admit I am wrong here. 1.2 Gigs with kernel and base on FreeBSD 13.2-RELEASE amd64
2GB is sufficient.

Phishfry · May 10, 2023

tripwire would frag mtree and email you a report about it.

Just kidding. tripwire is only defensive. Its reports are awesome.

Sergei_Shablovsky · May 11, 2023

Phishfry said:
I want to highly recommend security/tripwire for firewall usage.

It takes a bit to setup but once tuned it is the ultimate sentry.

Thank You for suggestions!

For what exactly files/type of files/directories You using this tripwire?

Sergei_Shablovsky · May 11, 2023

Phishfry said:
tripwire is only defensive. Its reports are awesome.

Possible example of reports? (if not on NDA)

Sergei_Shablovsky · May 11, 2023

alexseitsinger said:
A ZFS may be underutilized on a gateway.

And also need to note that for REALLY GREAT & FAST work of ZFS (ZFS RAID I mean, and on 3+Tb each HDD) better to have 64/128Gb RAM.

But of course, this is not the case of FreeBSD firewall.

Phishfry · May 11, 2023

Code:

tripwire -m c
Parsing policy file: /usr/local/etc/tripwire/tw.pol
*** Processing Unix File System ***
Performing integrity check...
Wrote report file: /var/db/tripwire/report/x9srl-20230511-062749.twr


Open Source Tripwire(R) 2.4.3.7 Integrity Check Report

Report generated by:          root
Report created on:            Thu May 11 06:27:49 2023
Database last updated on:     Never

===============================================================================
Report Summary:
===============================================================================

Host name:                    x9srl
Host IP address:              192.168.1.1
Host ID:                      None
Policy file used:             /usr/local/etc/tripwire/tw.pol
Configuration file used:      /usr/local/etc/tripwire/tw.cfg
Database file used:           /var/db/tripwire/x9srl.twd
Command line used:            tripwire -m c

===============================================================================
Rule Summary:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                       Severity Level    Added    Removed  Modified
  ---------                       --------------    -----    -------  --------
  Tripwire config                 100               0        0        0       
* megarouter config               100               0        0        6       
* Root config files               100               0        0        3       

Total objects scanned:  40
Total violations found:  9

===============================================================================
Object Summary:
===============================================================================

[SNIP]

Phishfry · May 13, 2023

These are the files I consider indicators something is wrong.
Notice above I have 3/6 files that were tripped. That was just me updating.

Code:

## twpolicy.txt
[SNIP]
(rulename = "Root config files", severity = 100 )
{
        /root                           -> $(SEC_CRIT);
        /root/.cshrc                    -> $(SEC_CONFIG);
        /root/.login                    -> $(SEC_CONFIG);
        /root/.history                  -> $(SEC_CONFIG);
        /root/.profile                  -> $(SEC_CONFIG);
        /root/.ssh/known_hosts          -> $(SEC_CONFIG);
}
#
(rulename = "megarouter config", severity = 100)
{
        /etc/rc.conf                                    -> $(CheckAll);
        /boot/loader.conf                               -> $(CheckAll);
        /etc/sysctl.conf                                -> $(CheckAll);
        /etc/pf.conf                                    -> $(CheckAll);
        /usr/local/etc/dnsmasq.conf                     -> $(CheckAll);
        /etc/ssh/sshd_config                            -> $(CheckAll);
        /etc/ssh/ssh_config                             -> $(CheckAll);
        /etc/fstab                                      -> $(CheckAll);
        /etc/ttys                                       -> $(CheckAll);
        /etc/resolv.conf                                -> $(CheckAll);
        /usr/local/etc/dnsmasq.d/dnsmasq.blacklist.txt  -> $(CheckAll);
}
[SNIP]

Al Seitsinger · May 13, 2023

Phishfry said:

These are the files I consider indicators something is wrong.
Notice above I have 3/6 files that were tripped. That was just me updating.

Code:

## twpolicy.txt
[SNIP]
(rulename = "Root config files", severity = 100 )
{
        /root                           -> $(SEC_CRIT);
        /root/.cshrc                    -> $(SEC_CONFIG);
        /root/.login                    -> $(SEC_CONFIG);
        /root/.history                  -> $(SEC_CONFIG);
        /root/.profile                  -> $(SEC_CONFIG);
        /root/.ssh/known_hosts          -> $(SEC_CONFIG);
}
#
(rulename = "megarouter config", severity = 100)
{
        /etc/rc.conf                                    -> $(CheckAll);
        /boot/loader.conf                               -> $(CheckAll);
        /etc/sysctl.conf                                -> $(CheckAll);
        /etc/pf.conf                                    -> $(CheckAll);
        /usr/local/etc/dnsmasq.conf                     -> $(CheckAll);
        /etc/ssh/sshd_config                            -> $(CheckAll);
        /etc/ssh/ssh_config                             -> $(CheckAll);
        /etc/fstab                                      -> $(CheckAll);
        /etc/ttys                                       -> $(CheckAll);
        /etc/resolv.conf                                -> $(CheckAll);
        /usr/local/etc/dnsmasq.d/dnsmasq.blacklist.txt  -> $(CheckAll);
}
[SNIP]

What about replaced binaries? time changes? Modifications to (master.)passwd/groups? Changes to libc? Typically, most of the filesystem should be watched IMO.

Sergei_Shablovsky · May 25, 2023

gpw928 said:
So you want the file system containing the logs isolated from all other activity. With USF, this means a separate file system. With ZFS this means a separate file system in a separate (not zroot) pool -- because all file systems in a ZFS pool share a common pool of spare available disk space -- and filling one file system fills them all.

Again thank You for detailed explaining.

How to realize this (I mean creating the 2 file systems in case UFS, and separate file system on a separate pool (not zroot)) in automatic mode in /etc/installerconfig script for bsdinstall utility?
(Code example)

Thank You so much!

Phishfry · May 31, 2023

alexseitsinger said:
Modifications to (master.)passwd/groups?

Yea I might need to tighten that up after reading this:

Valid Accounts, Technique T1078 - Enterprise | MITRE ATT&CK®

attack.mitre.org

How to stop some one from creating a reverse shell??
How do you lockdown shells & creation of new ones.

Sergei_Shablovsky · May 28, 2024

Alex Seitsinger said:
A ZFS may be underutilized on a gateway.

Please, coul You be so please explain this in details ?
Thank You so much!