Solved pf blocks zfs send/receive?

[bxbzq]

Hi folks,

So, I schedule samba server zfs replication to a backup server, using crontab on the samba server:

30    22    *    *    1-6        root    zxfer -dFkPv -I keylocation,keyformat,pbkdf2iters -T root@192.168.1.151 -R zroot/jtanks/samba/store1 backup/samba

It's been working just fine. Today I was checking the backup server and found no snapshot was transferred from the samba server since couple of days ago. Been thinking about the reason then I remembered I enabled pf on the backup server on the same day the last snapshot was transferred. I believe it is the settings in pf.conf that is blocking the zfs replication. Checking the /etc/services, I didn't find any port or service pertaining to zfs send/receive therefore don't know what to change.

How do I configure pf to bring back the zfs replication? Can anyone point me to the right direction?

 

[mer]

I believe it is the settings in pf.conf that is blocking the zfs replication.

To verify this, disable pf on the receiving system and see if you get snapshots.

Then you'll need to post your rules/pf.conf in order for people to help.
But a guess on my part: if you have pf "default deny in" and nothing explicitly allowing from the sender that would block the incoming connection. A way to verify this would be add a allow in all quick from the sender near the top of your rules.

 

[bxbzq]

To verify this, disable pf on the receiving system and see if you get snapshots.

Then you'll need to post your rules/pf.conf in order for people to help.
But a guess on my part: if you have pf "default deny in" and nothing explicitly allowing from the sender that would block the incoming connection. A way to verify this would be add a allow in all quick from the sender near the top of your rules.

I'll try to disable pf later today. Here is what I have in /etc/pf.conf

ext_if = "re0"
client_out = "{ ssh, domain, ntp, http, https }"
icmp_types = "{ echoreq, unreach }"
table <bruteforce> persist
table <rfc6890> { 0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16        \
                  172.16.0.0/12 192.0.0.0/24 192.0.0.0/29 192.0.2.0/24 192.88.99.0/24    \
                  192.168.0.0/16 198.18.0.0/15 198.51.100.0/24 203.0.113.0/24        \
                  240.0.0.0/4 255.255.255.255/32 }
set skip on lo0
set block-policy return
set loginterface $ext_if
scrub in all fragment reassemble max-mss 1440
antispoof quick for $ext_if
block in quick on egress from <rfc6890>
block return out quick on egress to <rfc6890>
block all
anchor "blacklistd/*" in on $ext_if
pass in on $ext_if proto tcp to port { ssh }    \
    keep state (max-src-conn 5, max-src-conn-rate 3/1,    \
            overload <bruteforce> flush global)
pass out proto { tcp, udp } to port { 22, 53, 80, 123, 139, 443, 445, 3389 }
pass out inet proto icmp icmp-type $icmp_types
table <sshguard> persist
block in quick on $ext_if from <sshguard> label "SSH Guard"

 

[mer]

I think that block all is causing it. a quick check would be add something like after the anchor for blacklistd:

pass in quick from ip address of the sender

that would allow incoming traffic from samba server to initiate the connection.

 

[VladiBG]

If your server is in 192.168.0.0/16 network you should add pass in rule above "block in quick on egress from <rfc6890>" as mer already pointed out.

 

[yuripv79]

Nitpicking: zfs send/recv does NOT do anything network-related, it just generates the data stream/creates snapshot from it, so you need to look at whatever protocol is used by the tool that does the actual transfer of data streams between hosts (ssh, rsync, something else).

zfs send -> stdout
stdin -> zfs recv

Taking a quick look at the zxfer man page, it seems to use rsync and ssh, try allowing both.

 

[bxbzq]

Thanks for all the replies. Still need more clarification.

The samba server's ip is 192.168.1.254, so it is in 192.168.0.0/16 network. I tried both adding a line "pass in quick from server_ip" before "block in quick on egress from <rfc6890>" and after "anchor "blacklistd/*" in on $ext_if". Either one works. This confuses me.

I think that block all is causing it. a quick check would be add something like after the anchor for blacklistd:

pass in quick from ip address of the sender

that would allow incoming traffic from samba server to initiate the connection.

If your server is in 192.168.0.0/16 network you should add pass in rule above "block in quick on egress from <rfc6890>" as mer already pointed out.

If I read the zxfer man page correctly, in my zxfer command, the -S option is absent, -T is specified, then the transfer between samba server and backup server uses ssh. Meanwhile I have the line pass in on $ext_if proto tcp to port { ssh } ... in pf.conf. Why does it not work?

Nitpicking: zfs send/recv does NOT do anything network-related, it just generates the data stream/creates snapshot from it, so you need to look at whatever protocol is used by the tool that does the actual transfer of data streams between hosts (ssh, rsync, something else).

zfs send -> stdout
stdin -> zfs recv

Taking a quick look at the zxfer man page, it seems to use rsync and ssh, try allowing both.

 

[VladiBG]

The ip addresses from rfc6890 are filtered for announcements in BGP between internet providers but in the internal network of the provider he may use them usually 100.64.0.0/10 is used to route the public IP from the MPLS Provider edge (PE) router to the customer premises equipment (CPE) router. So you may see in many firewall configuration examples a rule which blocks rfc6890 on CPE WAN interface when this WAN interface have public IP address assigned. But if this interface have private IP address from rfc6890 networks then this network should be excluded and allowed for communication in the firewall rules otherwise you are filtering a traffic which is expected to come at that CPE WAN interface.

In your case the class C network 192.168.0.0/16 should not be blocked on the interface as it's used on that interface and the traffic that is coming from it is internal traffic from your LAN network. So you can either edit the table <rfc6890> and remove 192.168.0.0/16 from it OR add allow rules before the " block in quick on egress from <rfc6890>"