Is it possible to source-based routing without the use of PF?

GreenX

GreenX

The situation is as follows: there is a server for multiple services. Services put in jail (for example: monitoring zabbix). In jails, gray addresses and a separate interface for them, and internal networks. The main interface uses "white" ip address, but it is not so important.

It is important that: I do not like the fact that outbound traffic goes through the main interface, and returns through another. I know how it is possible to try to do with PF. But in my opinion, that the firewall does not have to deal with routing.

Therefore I find the right solution.
 
GreenX said:
It is important that: I do not like the fact that outbound traffic goes through the main interface, and returns through one another.
Click to expand...

Outbound traffic will always pass from your main interface. The same goes for inbound traffic.

If your concern is regarding intra-jails communications, then you can bind each jail to a different physical interface.
 
Outbound traffic will always pass from your main interface. The same goes for inbound traffic.
Click to expand...
Wrong. It depends on routing table and can be changed also by "ipfw fwd" rule. FreeBSD has support for multiple routing tables from 2007 year, see, for example:
http://www.mmacleod.ca/blog/2011/06/source-based-routing-with-freebsd-using-multiple-routing-table/

So the answer is yes, you could use PBR without PF, use setfib() and ipfw setfib ....

For example, my config of ISP's NAT-server:
Code: 
00100 allow ip from any to any via lo0
00200 allow ip from 10.1.0.0/24 to 10.1.0.0/24
00400 setfib 1 ip from any to any via vlan500 // RosTelecom
00500 setfib 1 ip from table(2) to any // RosTelecom
00600 setfib 1 ip from 212.58.xxx.248/29 to any // RosTelecom
00700 setfib 1 ip from any to 212.58.xxx.248/29 // RosTelecom
00800 fwd 212.58.xxx.249 ip from 212.58.xxx.250 to any xmit vlan720 // My traffic to RosTelecom
00900 setfib 1 ip from 94.142.xxx.0/24 to any // RosTelecom's real ips
01000 count log logamount 29999 tcp from any to any dst-port 25 setup recv vlan100 // Dumb antispam rule
01200 allow tcp from any to me dst-port 22 setup keep-state // SSH
01400 allow tcp from table(11) to me dst-port 20,21,80 setup keep-state // FTP and HTTP for trusted clients
01500 allow tcp from any to me dst-port 80 setup recv vlan100 keep-state // Internal HTTP
01600 allow tcp from any to me dst-port 443 setup keep-state // HTTPS for any clients (ACLs in Apache, for payments)
02000 fwd 127.0.0.1,3128 tcp from any to 212.193.xxx.77,92.241.xxx.120 dst-port 80 recv vlan100 // TProxy for real xxx.ru and gismeteo.ru
03000 skipto 6000 ip from table(1) to any // Skip deny rules for allowed ips
03100 unreach filter-prohib ip from any to not me in recv vlan100 // Verbose deny for internal lan
04000 netgraph 4000 ip from any to me recv vlan720 // Inside NAT
04100 netgraph 4100 ip from any to 212.58.xxx.251 recv vlan500 // Inside NAT
06000 pipe tablearg ip from any to table(4) xmit vlan100
06100 pipe tablearg ip from table(5) to any xmit vlan720
06200 pipe tablearg ip from table(5) to any xmit vlan500
07000 netgraph 7000 ip from 10.1.0.0/16 to any xmit vlan720 // Outside NAT
07100 netgraph 7100 ip from 10.1.0.0/16 to any xmit vlan500 // Outside NAT
08000 allow ip from me to any xmit vlan720 // Allow all from me
08100 allow ip from 212.58.xxx.248/29,94.142.xxx.0/24 to any xmit vlan500
08150 allow tcp from me to any setup keep-state // Tcp from me
08160 allow udp from any to me keep-state // Open my udp
08170 allow udp from me to any keep-state // Open my udp
08200 allow ip from any to any established
08300 allow ip from table(1) to any
08400 allow ip from any to table(1)
08500 allow icmp from any to any icmptypes 0,3,4,8,11
65530 deny log logamount 100 ip from any to any
65535 allow ip from any to any
 
Imagine that the service is in jails does not know anything about the existence of any other network interface except the main for it! Is that possible? It is!
This is what I want to do. Whatever, the system worked in the prison about their routing tables.
While I stood on the fact that "route flush" does not completely clears the routing table...
 
gkontos said:
He is asking about jails.
Click to expand...

I've looked into /etc/rc.d/jail I see that all IP addresses assigned to any jail are assiged with /sbin/ifconfig and can be used by any standard or jailed process. The jail framework is able to limit the network access for a specific process and its descendants, but currently it's using hosts' routing mechanism.
This assymetry problem has nothing specific to do with the jail method of running processes, it's related to host's routing table. Since a FreeBSD host runs a single kernel, it would be inappropriate to believe that it has multiple ip stack instances. It may be a useful feature to bind a jail to a specific network interface and with a fully complete ip stack instance, but the use of a such feature would be very limited, requiring extra overhead at the network's layer 2 and 3.
 
ecazamir said:
A more powerful routing mechanism, perhaps similar to iproute2
Click to expand...

Yes, but what for? Jails are statically connected therefore there is no need for any sort of dynamic routing.
 
This is required for multihomed systems. It may be possible to handle this with current tools, a special section in handbook is welcome.
The expected behaviour is: "on a multihomed system, regardless of the default route, any incoming connection to be replied through the same path". Use of FIBs require multiple process instances, because a process is using only one FIB.
 
ecazamir said:
The expected behaviour is: "on a multihomed system, regardless of the default route, any incoming connection to be replied through the same path".
Click to expand...

I haven't tried it on a BSD box yet but I would expect this to be the default behavior.
 
No, this it's not the default behaviour. Linux, Windows and FreeBSD, with default routing is replying through the same connection only if the source belongs to a directly-connected subnet. Otherwise, the packet leaves the system through the interface which has the default gateway configured and selected.
For processes running only one instance (no setfib or similar FIB enforcement), this can be overriden with pf() on FreeBSD, or with iproute2 and iptables on Linux. iproute2 allows next-hop selection (default route override) by a number of criteria, including the source address in the packet and the connection state.
iproute2 allows this for any process/socket, even if it's using multiple routing tables at one time. The listening process does not have to be started 'with a specific FIB'.
FreeBSD should have a FIB/connection state pairing method, iproute2 and iptables has it since many years ago.
Real-world example:
Code: 
server
   interfaces
       eth0 - wan provider 1 (world 1, sub-world 1, preferred default route)
       eth1 - wan provider 2 (world 2, sub-world 2, default route avaliable)
       eth2 - wan provider 3 (world 3, sub-world 3, default route avalialbe)
       ...
       ethN
   processes
       sshd
       httpd
   ip_addresses
       172.16.0.0/24 (public addressable range 1, provider independent)
       172.16.1.0/24 (public addressable range 2, provider independent)
'world' is the entire internet, 'sub-world' is the provider IP address space.

If httpd must be started with many instances (for virtual ip-based hosts), the link having the preferred default route will be overloaded if pf's route override mechanism or iproute2 is not used, if the http client belongs to world X, using the link only for the incoming packets.

If there is a routing protocol like BGP configured between this router and sub-world1/sub-world2 and sub-world3, then each client belonging to sub-world X should have it's connection replied through the arriving interface, but the ISP's IP range is usually limited.
 
ecazamir said:
No, this it's not the default behaviour. Linux, Windows and FreeBSD, with default routing is replying through the same connection only if the source belongs to a directly-connected subnet. Otherwise, the packet leaves the system through the interface which has the default gateway configured and selected.
Click to expand...

When it comes to more than basic routing issues, my only experience so far is Cisco and Checkpoint Firewalls. Some Cisco routers as well mainly because the job has to be done now rather than to expect from a network department.

I can tell you for sure that this doesn't happen in either case regarding those two as long as the source is known in their routing table.

I believe that the same goes for any OS but what you just described, a source not being directly connected, certainly implies that some sort of dynamic routing protocol must be used.

In any case my confusion was caused because I didn't understand that the OP wanted to extend his jail services beyond the scope of a LAN. Which IMO is wrong, security wise.

Best regards,
George
 
Right, the packet will leave the system according to the routing table. On the example above I assumed that services run on a set of addresses reachable from any WAN link. Some users want to try another configuration: dual WAN links, with different IP space for each, for backup purposes. This setup works fine until the link having the default route fails.

Cisco routers with static configurations perform in a simiar way, if no monitors are configured. IMO it's better to put a web server into a jail instead of letting it run unrestricted.
 
Sorry for the long silence. I see that lack of understanding of the need, creates confusion for what is right. The simplest example: the system has two interfaces. The first uses the address 10.0.0.0/24, it is also used as the default route. The second uses the address of 10.0.1.0/24. But the external network does not know or not should know the route to another network or does not address filters are not owned by it.
 
gkontos said:
Maybe I got this wrong here but I haven't really figured out what the OP wants to do.
Click to expand...
So you don't actually know what the question is, but you think you need to provide answers? Perhaps you should either take the time to try and understand what the question is, or keep quiet and let those that do understand provide correct and relevant information.
 
You must log in or register to reply here.
Back
Top