Solved VNET Jails pf.conf unbound nginx port forwarding

[freebuser]

I am posting in this forum after a long time, but a regular browser.

I have setup a vnet jail for unbound and also running nginx.
The vnet jail is on epair1a/b with 10.0.0.211/10.0.0.198 ips. host ip is 10.0.0.112
Host has lagg0 with a 4 port NIC and vnet jail is bridged to bge0 which is an onboard NIC.

From what I understand, the jail should be able to communicate to the local network 10.0.0.0/24 without any need of pf rules on the host.

But it doesn't, the unbound only works when I include the rule in pf.conf

 pass log in

and

 pass log out

.

but even with the above codes I am unable to access the vnet jail's port 80 even if it is properly forwarded on the Mikrotik router (the host's port 80/443 works without any issues).

So, is it required for me to NAT/RDR ports in the pf for the vnet jail to work?

If so, then to pass in/out any ports to the jail which $net_if I use epair1a/b or bge0?

Jail's etc/rc.conf

postfix_enable="YES"
sendmail_enable="NONE"
dovecot_enable="YES"
rspamd_enable="YES"
redis_enable="YES"

unbound_enable="YES"
## (if from ports, use) unbound_enable="YES"
dnscrypt_proxy_enable="YES"
dnscrypt_proxy_resolver="dnscrypt.eu-nl"
#dnscrypt_proxy_flags="-a 10.0.0.198:853"

nginx_enable="YES"
dccifd_enable="YES"

etc/pf.conf

int_if2="bge0"
int_if="lagg0"

#jail_if0="epair0a"
#jail_if1="epair1a"

serverip="10.0.0.112"

webserver_jail="10.0.0.193"
dnsserver_jail="10.0.0.198"
#webserver_gateway="10.0.0.211"
#dnsserver_gateway="10.0.0.210"

#PORT_JAIL_WWW="{8081}"

set block-policy return
set limit table-entries 400000

PORT_WWW="{25,143,465,587,993,80,8080,443,5203,6379,3690,445,6295,5001,5349,3478,8088,8448}"
PORT_NFS="{111,2049,845,744,650}"
PORT_IPERF="{5002}"
PORT_MYSQL="{3306}"
PORT_POUDRIERE="{8989}"
PORT_DNS="{53,853}"
PORT_WWW_JAIL="{80}"
### all incoming traffic on external interface is normalized and fragmented
### packets are reassembled.
scrub in on $int_if all fragment reassemble
scrub in on $int_if2 all fragment reassemble



#nat on $int_if inet from $jail_if0 to any -> ($int_if)
#rdr pass on $int_if inet proto tcp to ($int_if) port {8081} -> $webserver_jail
#rdr pass on $int_if2 inet proto tcp to ($int_if2) port 9443 -> $dnsserver_jail

### set a default deny everything policy.
block drop log on $int_if all

table <pfbadhost> persist file "/etc/pf-badhost.txt"
block in quick on egress from <pfbadhost>
block out quick on egress to <pfbadhost>


### exercise antispoofing on the external interface, but add the local
### loopback interface as an exception, to prevent services utilizing the
### local loop from being blocked accidentally.
set skip on lo0
set skip on bge0


antispoof for $int_if inet
antispoof for $int_if2 inet

### block anything coming from sources that we have no back routes for.
block in log from no-route to any

### block packets that fail a reverse path check. we look up the routing
### table, check to make sure that the outbound is the same as the source
### block packets that fail a reverse path check. we look up the routing
### table, check to make sure that the outbound is the same as the source
### it came in on. if not, it is probably source address spoofed.
block in log from urpf-failed to any

### drop broadcast requests quietly.
block in log quick on $int_if from any to 255.255.255.255
block in log quick on $int_if2 from any to 255.255.255.255

### block probes that can possibly determine our operating system by disallowing
### certain combinations that are commonly used by nmap, queso and xprobe2, who
### are attempting to fingerprint the server.
### * F : FIN  - Finish; end of session
### * S : SYN  - Synchronize; indicates request to start session
### * R : RST  - Reset; drop a connection
### * P : PUSH - Push; packet is sent immediately
### * A : ACK  - Acknowledgement
### * U : URG  - Urgent
### * E : ECE  - Explicit Congestion Notification Echo
### * W : CWR  - Congestion Window Reduced
block in quick on $int_if proto tcp flags FUP/WEUAPRSF
block in quick on $int_if proto tcp flags WEUAPRSF/WEUAPRSF
block in quick on $int_if proto tcp flags SRAFU/WEUAPRSF
block in quick on $int_if proto tcp flags /WEUAPRSF
block in quick on $int_if proto tcp flags SR/SR
block in quick on $int_if proto tcp flags SF/SF

#table <pfbadhost> persist file "/etc/pf-badhost.txt"
#block in quick on egress from <pfbadhost>
#block out quick on egress to <pfbadhost>

### keep state on any outbound tcp, udp or icmp traffic. modulate the isn of
### outgoing packets. (initial sequence number) broken operating systems
### sometimes don't randomize this number, making it guessable.
pass on $int_if proto { tcp, udp, icmp } from any to any modulate state
pass on $int_if2 proto { tcp, udp, icmp } from any to any modulate state
##pass log on $jail_if0 proto { tcp, udp, icmp } from any to any modulate state
##pass log on $jail_if1 proto { tcp, udp, icmp } from any to any modulate state

### normally, a client connects to the server and we handshake with them, then
### proceed to exchange data. by telling pf to handshake proxy between the client
### and our server, tcp syn flood attacts from ddos become uneffective because
### a spoofed client cannot complete a handshake.

### set a rule that allows inbound ssh traffic with synproxy handshaking.
#pass in on $int_if proto tcp from any to any port {XXXX} flags S/SA synproxy state
#pass in log(all) on $int_if proto tcp from 10.0.0.0/24 to $serverip port 21 flags S/SA synproxy state

#pass in log(all) on $int_if proto tcp from any to  $serverip port 21 flags S/SA synproxy state
pass in on $int_if proto tcp from 10.0.0.0/24 to $serverip keep state
#pass in on $int_if2 proto tcp from 10.0.0.0/24 to $serverip2 keep state

### set a rule that allows inbound www traffic with synproxy handshaking.
pass in on $int_if proto tcp from any to any port www flags S/SA synproxy state

### set a rule that allows inbound www traffic with synproxy handshaking.
pass in on $int_if proto tcp from any to any port www flags S/SA synproxy state
pass in on $int_if2 proto tcp from any to any port www flags S/SA synproxy state

### setup a table and ruleset that prevents excessive abuse by hosts
### that attempt to brute force the ssh daemon with repeated requests.
### any host that hammers more than 3 connections in 5 seconds gets
### all their packet states killed and dropped into a blackhole table.
table <ssh_abuse> persist
block in quick from <ssh_abuse>
pass in on $int_if proto tcp to any port {XXXX,993,465,143,110} flags S/SA keep state (max-src-conn 10, max-src-conn-rate 3/5, overload <ssh_abuse> flush)
pass in on $int_if2 proto tcp to any port XXXX flags S/SA keep state (max-src-conn 10, max-src-conn-rate 3/5, overload <ssh_abuse> flush)

# Default TCP policy
   pass in quick on $int_if proto TCP from any to $serverip port {2209,3306,9000} flags S/FSRA synproxy state
##   pass in quick on $int_if2 proto TCP from any to $serverip2 port {2209,3306,9000} flags S/FSRA synproxy state
   pass in quick on $int_if proto TCP from any to $serverip port $PORT_POUDRIERE flags S/FSRA synproxy state
##   pass in quick on $int_if2 proto TCP from any to $serverip2 port $PORT_POUDRIERE flags S/FSRA synproxy state

###   pass log on  $int_if proto TCP from any to $webserver_jail port $PORT_JAIL_WWW  flags S/FSRA synproxy state
###   pass log on  $int_if proto TCP from any to $dnsserver_jail port $PORT_WWW  flags S/FSRA synproxy state

   pass in on $int_if proto TCP from any to $serverip port $PORT_WWW  flags S/FSRA synproxy state
##   pass in on $int_if2 proto TCP from any to $serverip2 port $PORT_WWW  flags S/FSRA synproxy state
   pass in quick on $int_if proto TCP from any to $serverip port $PORT_NFS  flags S/FSRA synproxy state
   pass in quick on $int_if proto TCP from any to $serverip port $PORT_IPERF  flags S/FSRA synproxy state

# Default UDP policy
  pass in quick on $int_if proto UDP from any to any port {5349} keep state
  pass in quick on $int_if proto UDP from any to any port $PORT_NFS keep state
  pass out quick on $int_if from $serverip to any keep state ###Check this
##  pass out quick on $int_if2 from $serverip2 to any keep state ###Check this

# emerging threats list
#create empty table in memory
table <emerging-block-ips> persist

# in filter section block all ip in <et_block_ip>
block in on $int_if from <emerging-block-ips>
block in on $int_if2 from <emerging-block-ips>

# naxsi config 127.0.0.1:4242
#  pass in quick on $int_if proto TCP from any to any port 4242
#  pass out quick on $int_if proto TCP from any to any port 4242

#pass in log from { self, $jail_if0} to any keep state
###pass in log proto tcp to $webserver_jail # port $PORT_JAIL_WWW
###pass out log proto tcp from $webserver_jail to any # port $PORT_JAIL_WWW

pass in log
pass out log

#pass out proto { tcp udp } from any to any port $PORT_DNS

 

[freebuser]

Because I have included bge0 (which is VNET jail's NIC) in the host's pf.conf it started to block on that interface too. Once I removed the "bge0" and all references it works as expected.