pr and bridges and squids, oh my!

FzZzT

Member


Messages: 34

Hello,

I've read a number of other threads and resources (here and elsewhere) but I can't seem to get the correct combination of things to make my scenario work. Some info seems to be outdated or I'm not sure how to fit it in. Maybe it just isn't possible. Hopefully this isn't completely duplicated with another post, but here we go...

I'm trying to configure a device that I have to act as a transparent filtering bridge and transparent outbound web proxy. I've gotten the bridge working, but haven't been able to get the forwarding to work to send web requests (http and https) to squid. I've tried all sorts of combinations of rdr, route-to and filter lines I don't really know how it's even working at all at this point. The bridge does work and filter, but nothing gets routed to squid on port 3128. The device has three interfaces and I'm using two. igb1 is connected to a switch which is connected to my PC, and igb0 is connected to my Internet router.

I have a Node web app that I made (running on port 3000) which manages pf address tables. I don't think this is interfering or preventing the squid redirection, but anything is possible. The rules from that are included.

Any help figuring this out would be appreciated. I prefer pf, but if that's a problem for doing this on FreeBSD, I could switch to something else. I could also swap out squid for something else. I don't really want to swap out FreeBSD for something else, though.

Code:
# uname -a
FreeBSD sentinel 12.1-RELEASE-p6 FreeBSD 12.1-RELEASE-p6 GENERIC  amd64
#
# sysctl net.link.bridge
net.link.bridge.ipfw: 0
net.link.bridge.allow_llz_overlap: 0
net.link.bridge.inherit_mac: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 0
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_bridge: 1
net.link.bridge.pfil_onlyip: 1
#
# grep -vE '^(#|$)' /etc/pf.conf
set skip on lo0
pass in quick proto tcp from any to any port { 22, 3000 } keep state
pass quick on bridge0 inet from 192.168.254.0/24 to 192.168.254.0/24
pass quick on bridge0 inet proto tcp from any to any port { 80, 443 } keep state
include "/usr/local/etc/sentio/pf.conf"
block drop log all label "default-deny"
#
# grep -vE '^(#|$)' /etc/pf.conf
set skip on lo0
pass in quick proto tcp from any to any port { 22, 3000 } keep state
pass quick on bridge0 inet from 192.168.254.0/24 to 192.168.254.0/24
pass quick on bridge0 inet proto tcp from any to any port { 80, 443 } keep state
include "/usr/local/etc/sentio/pf.conf"
block drop log all label "default-deny"
#
# pfctl -s rules
pass quick on bridge0 inet proto tcp from any to any port = http flags S/SA keep state
pass quick on bridge0 inet proto tcp from any to any port = https flags S/SA keep state
pass in quick proto tcp from any to any port = ssh flags S/SA keep state
pass in quick proto tcp from any to any port = 3000 flags S/SA keep state
pass quick on bridge0 inet from 192.168.254.0/24 to 192.168.254.0/24 flags S/SA keep state
block drop quick on bridge0 inet proto udp from any to any port = 1900
pass quick on bridge0 inet proto udp from any to <allow-dst-udp-123> port = ntp keep state
pass quick on bridge0 inet proto udp from any to <allow-dst-udp-53> port = domain keep state
pass quick on bridge0 inet proto udp from any to <allow-dst-udp-5998> port = 5998 keep state
pass quick on bridge0 inet proto udp from any to <allow-dst-udp-any> keep state
pass quick on bridge0 inet proto igmp from any to <allow-dst-igmp-any> keep state
block drop log all label "default-deny"
#
# ifconfig
igb0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0d:b9:43:44:c8
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e523bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0d:b9:43:44:c9
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e523bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0d:b9:43:44:ca
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:94:57:4d:bb:00
        inet 192.168.254.65 netmask 0xffffff00 broadcast 192.168.254.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: igb2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 3 priority 128 path cost 2000000
        member: igb1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 2000000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
        groups: pflog
 

mjollnir

Daemon

Reaction score: 700
Messages: 1,154

First try your setup without the filtering/restricting sections of the packet filter, instead enable only the redirecting rules for the transparent proxy & NAT. There's been another thread here on the forum about transparent HTTP proxy. You'll find it quickly.
 

wolffnx

Well-Known Member

Reaction score: 103
Messages: 445

you see the squid error log? maybe there is some clue
 
OP
FzZzT

FzZzT

Member


Messages: 34

There is nothing in the squid log, thats the problem, the redirect doesn't seem to work.

Oops I just realized that I didn't paste the conf with the rdr lines, ugh.

So I did some more experimenting, and if all I have is "block log all" it appears as though everything is only traversing bridge0:

Code:
# tcpdump -netti pflog0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
1597186150.160753 rule 1/0(match): block in on bridge0: fe80::fe2b:b2ff:fe04:350 > ff02::1: ICMP6, router advertisement, length 24
1597186151.301674 rule 1/0(match): block in on bridge0: 192.168.254.69.53835 > 192.168.254.254.53: 47889+ A? ocsp.pki.goog. (31)
[...]
I didn't see any traffic at all from my int/ext interfaces (igb1 and igb2), and adding rules (rdr or pass) for those doesn't seem to do anything.

I tried these lines for the redirect, using netcat to listen for packets, but again nothing ever came through:

Code:
#rdr pass inet proto tcp from $lan to any port { 80,443 } -> 127.0.0.1 port 3129

#rdr on bridge0 proto tcp from $lan to any port { 80, 443 } -> 127.0.0.1 port 3129

#rdr on $int_if inet proto tcp from $lan to any port { 80, 443 } -> 127.0.0.1 port 3129
#pass in quick on $int_if route-to lo0 inet proto tcp from $lan to 127.0.0.1  port 3129 keep state

#rdr inet proto tcp from $lan to any port { 80, 443 } -> 127.0.0.1 port 3129
Using the "rdr on bridge0" line I see these, but nothing in netcat on localhost:

Code:
# tcpdump -neti pflog0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
rule 1/0(match): rdr in on bridge0: 192.168.254.69.65365 > 127.0.0.1.3129: Flags [S], seq 2776708024, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
rule 1/0(match): rdr in on bridge0: 192.168.254.69.65366 > 127.0.0.1.3129: Flags [S], seq 2163167577, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
It seems like having net.link.bridge.pfil_member=0 might have affected it. Changing that to 1 now has blocked packets logged on igb1. So I guess I will try a bunch of tests again and see what changes...

With this combo, and pfil_member=1, I finally get some gibberish in netcat (which is probably the SSL handshake):

Code:
rdr log on $int_if proto tcp from $lan to any port { 80, 443 } -> 127.0.0.1 port 3129
pass in quick on $int_if route-to lo0 inet proto tcp from $lan to 127.0.0.1 port 3129 keep state
Restarting squid finally gets some packets:

Code:
==> /var/log/squid/access.log <==
1597187050.511      1 192.168.254.69 NONE/400 3700 NONE error:invalid-request - HIER_NONE/- text/html
1597187050.513      1 192.168.254.69 NONE/400 3700 NONE error:invalid-request - HIER_NONE/- text/html
1597187050.533      1 192.168.254.69 NONE/400 3892 GET /success.txt - HIER_NONE/- text/html
1597187050.536      0 192.168.254.69 NONE/400 3892 GET /success.txt - HIER_NONE/- text/html
Now to figure out how to get that piece working properly...
 
OP
FzZzT

FzZzT

Member


Messages: 34

Alright, getting closer. After a lot more fiddling I was able to get requests to show up in Squid, but now it's in some sort of loop. From reading online it sounds like there is still some sort of redirection issue...

Code:
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
rule 0/0(match): pass in on igb1: 192.168.254.69.62789 > 127.0.0.1.3127: Flags [S], seq 2553426832, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
2020/08/12 21:33:54 kid1| WARNING: Forwarding loop detected for:
GET / HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Via: 1.1 sentinel (squid/4.12)
X-Forwarded-For: 192.168.254.69
Cache-Control: no-cache
Connection: keep-alive
Host: bash.org
Code:
# grep -vE '^($|#)' /etc/pf.conf
ext_if="igb2"
int_if="igb1"
lan="192.168.254.0/24"
rdr log on $int_if proto tcp from $lan to any port 80 -> 127.0.0.1 port 3127
pass in log quick on $int_if route-to lo0 inet proto tcp from $lan to 127.0.0.1 port 3127 keep state
pass in quick on lo0 inet proto tcp from $lan to 127.0.0.1 port {3127,3128} keep state
pass out log quick inet proto tcp from 127.0.0.1 to any port 80 keep state
pass in quick on $int_if inet proto tcp from $lan to any port 443 keep state
pass in quick on $int_if inet proto udp from $lan to any port 53 keep state
pass out quick on bridge0 inet proto udp from $lan to any port 53 keep state
pass quick inet proto tcp from any to any port {22,443} keep state
pass quick on lo0
block log all
# grep -vE '^($|#)' /usr/local/etc/squid/squid.conf
acl whitelist url_regex "/usr/local/etc/sentio/urls"
http_access allow whitelist
http_access deny all
http_port 3128
http_port 3127 intercept
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all
https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=8MB cert=/usr/local/etc/squid/ssl_cert/cs7_squid_ca.pem
coredump_dir /var/squid/cache
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320
visible_hostname sentinel
 
OP
FzZzT

FzZzT

Member


Messages: 34

Welp I've tried transparent, intercept and nothing for the http_port line in squid.conf but I can't figure out why the forwarding loop issue is happening...
 
Top