ezjail, setfib and loopback connectivity

[pacija]

Hi,

I'm fairly familiar with ezjail, but quite a n00b to setfib concepts, hopefully someone can help me out.

I have a problem with services on loopback interfaces in jails which use separate fib - they don't work for me.

I've been configuring my ezjails as instructed in https://www.freebsd.org/doc/handbook/jails-ezjail.html, binding them to cloned lo1 interface with loopback address of 127.0.1.X (X being different for each jail) and physical interface with public IP address of 193.53.106.X. Jails serve clients from real IP address, but also use loopback interface for internal services (e.g. redis database).

Above works great with single fib, but now I got server with multiple network interfaces, and I'd like to serve both DMZ and LAN clients, on separate networks which can't communicate directly via jail host.

loader.conf:

net.fibs=4
net.add_addr_allfibs=0

rc.conf:

cloned_interfaces="lo1 lo2"
static_routes="dmz dmzd nix nixd"
route_dmz="-net 193.53.106.0/24 -interface bce0 -fib 1"
route_dmzd="default 193.53.106.254 -fib 1"
route_nix="-net 10.30.19.160/27 -interface bce2 -fib 2"
route_nixd="default 10.30.19.190 -fib 2"

corresponding ezjail's jail config:

export jail_somejail_fib="1"

As I said, service listening on physical interface appears to be working fine, but the one on loopback interface does not. I guess it's something about static routes for loopback interfaces but I struggle to figure out what.

Any help appreciated.

 

[SirDice]

Jails don't have a loopback interface. Use file sockets for IPC.

 

[pacija]

SirDice thank you for the tip, that's definitely _a_ solution, but not _the_ solution for me

Should I conclude from your answer that cloned loopback interfaces in jails that use different fibs aren't possible, aren't preferred way to go, or just that you don't know?

 

[SirDice]

I rarely use FIBs, I just bind my jails to lo1 and accept the fact there's no local loopback.

 

[pacija]

Well, I also bind my jails to lo1. I'm not quite sure what you mean by not having local loopback.

In non-fib setup, my jail has two interfaces with addresses (as seen by ifconfig from within the jail) - 127.0.1.132 on lo1 and 193.53.106.132 on bce0. I am successfully serving apache from 193.53.106.132, which successfully contacts redis at 127.0.1.132. I'm giving apache and redis for example, but have a myriad of similar setups - unifi5 on public and mongodb on looback etc.

If I move my jail to fib1, it shows same interfaces as in non-fib setup. The problem is that public one works fine, but internal does not. Switching from IP to sockets is the last possible solution.

Hope someone else chimes in as well...