Policy route and sshd on localhost?

Bleve97

Bleve97

Hi,
I'm seeking advice on the least worst way to solve a problem I have.

I have a client that has two connections to the Internet, a 5G modem/router and an aDSL modem/router. The 5G is for outgoing connections (www, email, all that jazz) and the aDSL service is for incoming SSH connections. These connections are to a FreeBSD box.
The 5G modem/router is from Telstra (the big telco in Australia if you're not familiar with them) and it has no port forwarding or static routing capability (in 2024, really?!) and they do not provide a static IP address no matter how much you ask them to do so. I feel kinda privilegded that it lets you change DHCP settings ... and turn DhCP off! Anyway, that's a rant for another place. I need a solution!

The aDSL service has a static IP and I can (and do!) port forward from it to the FreeBSD box for incoming SSH for management etc.

At the moment, I have both these devices on the same physical ethernet and IP network (192.168.0.0/24), the aDSL is 192.168.0.1, the 5G is 192.168.0.254 and the FreeBSD box is 192.168.0.2. As above, the aDSL device is port forwarding AND NAT'ing SSH for access to the FreeBSD box (currently FreeBSD 14.1). I could add an extra ethernet port to the FreeBSD box if required.

The FreeBSD box is working as a router for IPv4 and IPv6 (or it will, once I get it to behave with IPv6) and is running PF as a NAT service so devices on the inside (192.168.1.0/24) can get out. It's basically a firewall/router at the moment.

I need to be able to run almost everything via the 5G device, except ssh to and from the FreeBSD box (but not any other devices on the other side of the FreeBSD box). I can do a dodgey with a static route if I know where the ssh connection is coming from, but that only works as a proof of concept and will not scale to more than a device or two, and I need to support access in via SSH from IP addresses that I cannot predict.

I've gone down the rabbit hole of using PF's route-to, but groveling around whatever examples I can find online suggests that this will only work for traffic passing through the FreeBSD box, not arriving at it as its endpoint.
PF config looks like this :

Code: 
# NAT for Telstra 5G
#
# $Id: pf.conf,v 1.2 2024/11/20 23:27:40 carl Exp carl $
#
# [URL]https://docs.freebsd.org/en/books/handbook/firewalls/#firewalls-pf[/URL]

ext_if="re1"
int_if="re0"
localnet=$int_if:network
ADSL="192.168.0.1"


client_services = "{ ssh, smtp, smtps, submission, domain, www, \
        http, https, pop3, auth, pop3s, imap, imaps, ftp }"
udp_services = "{ domain, submission, ntp }"

#pass from re0:network to any port $ports keep state
# this is the NAT part, if the STOOPID Telstra G5 router could do static
# routes, this would not be necessary!
nat on $ext_if from $localnet to any -> ($ext_if)
#nat on $ext_if inet6 from $localnet to any -> ($ext_if)

# manage the GW from remote
# pass in inet proto tcp to $ext_if port ssh keep state
#pass in quick inet proto tcp on $ext_if from any to 192.168.0.2 port ssh tag MGT
pass in quick on $ext_if from any to 192.168.0.2 tag MGT
#pass out quick tagged MGT route-to ($ext_if $ADSL)

Various pokes at that ruleset have only resulted in "syntax error" on line 26, or 25, etc. But as above, I don't think this will work anyway.
Essentially I need a default route for sshd on the box itself, and no-where else.
I've had a quick look at some references to using multiple routing tables :
https://mmacleod.ca/2011/06/source-based-routing-with-freebsd-using-multiple-routing-table/
But that's old and I'm not sure that it's viable these days?

I'm considering spinning up a virtual machine on the box just to be an SSH entry point and giving it its own routing table, but that's pretty cumbersome for this, is there a better way?

Can anyone here suggest the least worst way to get this to work?
 
Thank you, I'm trying to track down a good (read, some howtos, not just a man page) for FIB stuff. Any hints?
 
I've done more grovelling, and found a guide-ish, and thus have done this :

in /etc/sysctl.conf :
net.fibs=2
Click to expand...

Then by hand I did this :
route -q -n add -inet 0.0.0.0/0 -interface re1 -fib 1
setfib 1 /usr/sbin/sshd -p 2021
Click to expand...

I changed my aDSL router to port forward to 2021 instead of 22, and in /varl/log/messages I see this when I try to SSH in to the box from the outside via the aDSL link :
arpresolve: can't allocate llinfo for x.y.z.q on re1
Click to expand...

presumably I've done something wrong

A tiny bit of debugging :
setfib 1 netstat -rn
Routing tables (fib: 1)

Internet:
Destination Gateway Flags Netif Expire
default link#2 US rel
Click to expand...

So it's -almost- working ...
but I'm stuck. This stuff has next to no doco anywhere that I've been able to find. I could RTFSC of the kernel I guess .. but C isn't my strongpoint!
"help"!
 
Did that, and for good measure, tried it on re1 as well (which has the 192.168.0.0/24 net on it), same problem. arpresolve is barfing on llinfo
 
i have no problem setting routes on in fib 1 (it just does not work from rc.conf) but it works from the cli or if runnig /etc/rc.d/routing again after boot
it looks like it tries needs to have the mac of the gw resolved and since the iface has no carrier at that time it fails
however it works ok after the boot completes
Code: 
ifconfig_re0="inet 10.1.1.106/24"
defaultrouter_fib1="10.1.1.2"
static_routes="s1"
route_s1="-net 10.1.1.0/24 -interface re0 -fib 1"
defaultrouter_fib1 fails during boot
 
setting net.add_addr_allfibs=1 in loader/sysctl.conf will make route_s1 obsolete but will still not set the fib 1 default route
 
Ok, I finally got it working, after a lot of search engine abuse, this thread was useful : https://forums.freebsd.org/threads/fib-and-route.53562/
What I did was in the end, pretty simple.
FreeBSD 14.2 amd64 (not that it should matter)

rc.local :
# Setup routing table for fib 1 to use the old aDSL service until we get openvpn working
# CB 20241206
#
# see also /etc/ssh/sshd_config.adsl

setfib 1 route add -net 192.168.8.0/24 -iface re2
setfib 1 route add -net 192.168.1.0/24 -iface re0
setfib 1 route add default 192.168.8.1

setfib 1 /usr/sbin/sshd -f /etc/ssh/sshd_config.adsl
Click to expand...

re2 is an ethernet adaptor which has the router on it that I want to use for SSH traffic in to this machine. It's connected to an aDSL modem/router that's got a static IP and is port forwarding ssh to 192.168.8.2 (the address on re2). It gets upset if you don't tell it about the networks on re2 and re0, you get errors from the route add default if you don't explicitly tell it about the interfaces before.

sshd_config.adsl has ListenAddress set to 192.168.8.2 so it's not listening on the other addresses the server has.
I didn't make any changes to sshd in rc.conf. I haven't rebooted the box yet to be 100% sure (not unless I'm onsite and can fix it if I need to, next week ...)

I think I needed to do this as well, I did it anyway ...
/etc/sysctl.conf :
net.fibs=2
Click to expand...
my "normal" ssh daemon is using a slightly modified sshd_config that has it explicitly listening to 192.168.1.1 and 192.168.0.2, which are its LAN and 5G modem link interfaces, instead of 0.0.0.0
I haven't looked at IPv6 on this yet, because the aDSL service doesn't provide IPv6.
Long term, I intend to replace this with a VPN to my home network which has static IPs so I can forward their traffic through it, and they can get rid of the aDSL service (and stop paying for it!).

The fib 1 routing table :
gw:/etc # setfib 1 netstat -rn
Routing tables (fib: 1)

Internet:
Destination Gateway Flags Netif Expire
default 192.168.8.1 UGS re2
127.0.0.1 link#4 UHS lo0
192.168.1.0/24 link#1 US re0
192.168.8.0/24 link#3 US re2
Click to expand...

And, well, it works!
There is a real scarcity of doco on fib and how to use it. Hopefully the above is helpful for the next poor sysadmin who has to do something like this. This is a shame (lack of doco) because this is really handy. Much nicer than trying to use firewalls (pf) and route-to and so on (which doesn't work in this case anyway)
 
You must log in or register to reply here.
Back
Top