PF unbound can’t forward requests from localhost to public DNS servers after PF reload

Hi all, I have been running the same rules for PF for two years and everything was working perfectly fine.

Problem comes after I upgraded my FreeBSD from 10.3-STABLE -> FreeBSD 11.2-STABLE. The local_unbound is not able to forward requests from localhost to public DNS server (8.8.8.8) after reloading the PF configurations. I am not sure if this is a bug related. Thanks in advance!

Code:
$ drill google.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 49252
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; google.com.    IN    A

;; ANSWER SECTION:
google.com.    300    IN    A    216.58.196.14

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 9 msec
;; SERVER: 127.0.0.1
;; WHEN: Wed Oct 24 15:28:18 2018
;; MSG SIZE  rcvd: 44

Code:
$ sudo service pf reload
Reloading pf rules.

Code:
$ drill google.com
Error: error sending query: Could not send or receive, because of network error

Code:
$ drill @8.8.8.8 google.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 40973
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; google.com.    IN    A

;; ANSWER SECTION:
google.com.    299    IN    A    216.58.196.14

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 11 msec
;; SERVER: 8.8.8.8
;; WHEN: Wed Oct 24 15:30:36 2018
;; MSG SIZE  rcvd: 44
 
Here is my /etc/pf.conf

Code:
### /etc/pf.conf
# macros
set limit { states 40000, frags 20000, src-nodes 20000 }

dia_if=ix0
lro_if=lagg0

mgt_sources="{ 192.168.1.0/24, 10.10.10.0/24 }"

tcp_any="{ 80, 443 }"
tcp_mgt="{ 22, 8080 }"
icmp_types="echoreq"

# options

set block-policy drop
set loginterface egress
set skip on lo

# match rules
block out log
block in log
pass out quick

antispoof quick for { lo }

block in on $dia_if
block in on $lro_if

pass in on egress inet proto tcp from $mgt_sources to (egress) port $tcp_mgt

pass in on $dia_if inet proto tcp from $mgt_sources to any port $tcp_mgt
pass in on $lro_if inet proto tcp from $mgt_sources to any port $tcp_mgt

pass in on $dia_if inet proto tcp from any to any port $tcp_any
pass in on $lro_if inet proto tcp from any to any port $tcp_any

# allow NFS port within local subnet
pass in on $lro_if inet proto tcp from 172.16.55.10
pass in on $lro_if inet proto udp from 172.16.55.10

pass in inet proto icmp all icmp-type $icmp_types
 
This one seems out of place because a few lines above it you have a set skip on lo

It was supposed to be commented out. Thanks for pointing that out.

But it doesn't seems to be the one causing the issue, the config has been there since long time ago and there's no issue with reloading the PF configuration.

It only happens after I upgraded the FreeBSD to v11.2-STABLE.

At first, I was suspecting could be due to some glitches in between during the upgrading process. So, I did a fresh install on another test machine, and it is having the same problem. It seems like a bug to me, but I can't be too sure on this.
 
Why run a developer snapshot in the first place though, why not just use the official RELEASE? Although I don't believe that my theory is true fact of the matter is that you should always keep possible bugs and hiatus in mind with developer snapshots, that's simply the nature of the beast.

Sure, STABLE is not bleeding edge and it seems a bit far fetched to assume this could be the reason for your problems, but it's still a possibility to keep in mind.

Anyway, I see you used
Code:
pass out quick
... but I can't help wonder if this sets up a state in the right way. I'm well aware that PF should do state tracking by default, but it seems that it doesn't in your example, making me to question that rule.

Try
Code:
pass out on any keep state
Just to rule out the possible issue.
 
Nothing wrong with using -STABLE. I've been running various -STABLE machines since 3.0 and I'm currently at 12-STABLE. Up until last weekend everything was on 11-STABLE and had been after it got branched off from HEAD (11.0-CURRENT at that time).

I like keeping track of new developments but don't want the risks associated with -CURRENT.
 
Try
Code:
pass out on any keep state
Just to rule out the possible issue.

I have been running this config and everything was fine up until I upgraded to FreeBSD 11.2-STABLE, also I was running FreeBSD 10.3-STABLE previously and I don't see any issue on reloading PF.

Also, your syntax seems to be wrong. Can't apply it to my config.
 
It's missing the interface; pass out on $some_interface {....}
 
Code:
# case 1
pass out on any keep state

# case 2
pass out on $mgt_sources keep state

Tried on both but getting syntax errors.

Code:
$ sudo pfctl -f /etc/pf.conf
/etc/pf.conf:24: syntax error
 
$mgt_sources is not an interface.

Code:
     on <interface>
           Specifies the interface the queue operates on.  If not given, it
           operates on all matching interfaces.
From pf.conf(4).
 
$mgt_sources is not an interface.

Code:
     on <interface>
           Specifies the interface the queue operates on.  If not given, it
           operates on all matching interfaces.
From pf.conf(4).


Alright, should be $lro_if instead. Haha... my bad.

Anyway, tried that but no avail. Now, I have to restart PF instead of reloading it every time for the config to be in place.

Am thinking to downgrade it to maybe 11.0-STABLE to see if the problem is there. :-/
 
There is no 11.0-STABLE. 11-STABLE is, at this time, 11.2-STABLE (Which is what you have now). FreeBSD 11.0-RELEASE is EoL and not supported any more.
 
Sorry, I cannot understand PF rules, therefore I ask. Is beside UDP 53 also TCP 53 open? 8.8.8.8 uses DNSSEC, and this goes over TCP 53 because UDP 53 does not work with the larger packets. Unbound is configured to use DNSSEC by default, therefore TCP 53 must be open on the firewall.
 
Anyway, tried that but no avail. Now, I have to restart PF instead of reloading it every time for the config to be in place.
Sorry for the typo up there, I do most of these things from mind.

I re-read the post and your shared output and I think I see what's wrong: you can query 8.8.8.8 manually using drill but you cannot contact the localhost it seems (assuming you have this set up in /etc/resolv.conf). Then I noticed this rule:

Code:
set skip on lo
However, the local interface is usually lo0, see also ifconfig. So now I wonder if this is causing your issues. This time I did verify with pf.conf(5) and there is no mention of lo, only lo0, as such my current conclusion.
 
I can't remember where I read it but if I remember correctly lo includes all loopback interfaces, like lo0, lo1, etc. But there doesn't appear to be a lo1 being used so it's definitely worth a try to set it to lo0 specifically.
 
Code:
set skip on lo0

This works perfectly, it seems my unbound able to forward request from localhost to public DNS servers even after I reload the PF config. Thank you so much!

However, I notice there's a consistent error on segmentation fault and it only happen on the first time after making the change in the /etc/pf.conf.

Code:
$ sudo service pf reload
Reloading pf rules.
Segmentation fault (core dumped)
$ sudo service pf reload
Reloading pf rules.
 
Code:
$ sudo service pf reload
Reloading pf rules.
Segmentation fault (core dumped)
$ sudo service pf reload
Reloading pf rules.
Could be a glitch. PF isn't really a service, so you could also consider using: # pfctl -Fa -f /etc/pf.conf.
 
Could be a glitch. PF isn't really a service, so you could also consider using: # pfctl -Fa -f /etc/pf.conf.

# pfctl -Fa -f /etc/pf.conf doing so will results same as doing # sudo service pf restart , no?

And it will kick out all the existing session, which I want to avoid all the time.
 
Code:
$ sudo pfctl -f /etc/pf.conf
Segmentation fault

$ sudo pfctl -f /etc/pf.conf

the segmentation fault is still consistent on the first change where it's only applicable on this:
Code:
set skip on lo0
, have to reload for the second time everytime. Weird...
 
looks like the problem is not caused by the code...

I have revert back the changes to
Code:
set skip on lo

and reloading the PF config twice make it works perfectly as it used to be.

Code:
$ sudo service pf reload
Reloading pf rules.
$ drill google.com
Error: error sending query: Could not send or receive, because of network error
$ sudo service pf reload
Reloading pf rules.
$ drill google.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 1143
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; google.com.    IN    A

;; ANSWER SECTION:
google.com.    299    IN    A    172.217.24.174

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 11 msec
;; SERVER: 127.0.0.1
;; WHEN: Sat Oct 27 11:03:25 2018
;; MSG SIZE  rcvd: 44

Anyone else is using FreeBSD 11.2-STABLE and have the same problem as I do?
 
I'm running 11.2-STABLE:
Code:
root@maelcum:~ # uname -a
FreeBSD maelcum.dicelan.home 11.2-STABLE FreeBSD 11.2-STABLE #0 r339481: Mon Oct 22 21:36:00 CEST 2018     root@build11:/usr/obj/usr/src/sys/MAELCUM  amd64
But I'm not seeing any of the issues you have:
Code:
root@maelcum:~ # pfctl -f /etc/pf.conf
root@maelcum:~ #
It just works, as it has been doing since a couple of years (not sure what this machine started with it's been running my firewall at home for quite a number of years).

Because it's a -STABLE version what's in your /etc/make.conf? How did you build world? Anything special in your kernel config?
 
I'm running 11.2-STABLE:
Code:
root@maelcum:~ # uname -a
FreeBSD maelcum.dicelan.home 11.2-STABLE FreeBSD 11.2-STABLE #0 r339481: Mon Oct 22 21:36:00 CEST 2018     root@build11:/usr/obj/usr/src/sys/MAELCUM  amd64
But I'm not seeing any of the issues you have:
Code:
root@maelcum:~ # pfctl -f /etc/pf.conf
root@maelcum:~ #
It just works, as it has been doing since a couple of years (not sure what this machine started with it's been running my firewall at home for quite a number of years).

Because it's a -STABLE version what's in your /etc/make.conf? How did you build world? Anything special in your kernel config?

I can't find any /etc/make.conf in my system here.

I'm running default kernel config I guess. Here's what I did:
Code:
cd /usr/src
make buildworld
make kernel

Are you able to do drill @127.0.0.1 google.com after the first pf config reload?

I have done a fresh install on a test machine previously using FreeBSD 11.2-STABLE and it is having the same issue. :( wonder which part have I done wrong.
 
Back
Top