How do you connect to a jailed server?

poorandunlucky

Well-Known Member

Reaction score: 30
Messages: 360

This is probably really stupid, and I'm in no way a FreeBSD newb, but I'm new to jails, I used to run everything on the main OS, but now I'm trying to setup things that are going to be open to the Internet (like NextCloud, a VPN, DNS) so I want them jailed... I've setup my jails no problem (thanks to bsdinstall jail [that's really neat, thanks]), and I know jail, jls, jexec, and I've tried to create pseudointerfaces like bge0.1 and binding the jail to that, I've also tried to bridge bge0.1 to bge0, and then I could get a DHCP lease for bge0 from my cable modem/router, but I can't get a DHCP lease from my jail, it broadcasts to 255.255.255.255 but gets no reply...

My router doens't support tagged VLANs, btw, if that could be part of my problem...

I've tried to setup forwarding rules in ipfw and that doesn't seem to work either...

I'm trying to open a HTTP connection to a jailed process on port 32400 (the NextCloud server)... I've loaded accf_data and accf_http, that wasn't it either...

Could anyone please tell me how you setup networking in a jail properly?

I can connect to it locally no problem, on the same machine, but I can't connect to my jail from the LAN... I need to open the jail to the LAN, and assigning a secondary IP to my interface doesn't work, and ideally, I'd want my jail to acquire a DHCP lease from the router...

Thanks in advance for any help,
 

Emrion

Well-Known Member

Reaction score: 107
Messages: 405

For the few things I know: you need to setup your network jail with VNET and add forwarding rules into the firewall. You have here some indications but they are for sysutils/iocage: https://iocage.readthedocs.io/en/latest/networking.html
Note that the FreeBSD kernel has VNET support (since 12.0 maybe?). No need to compile a custom kernel anymore.

However, if you can't reach your jail inside your own network, you have a problem. Normally, your network interface should be in promiscious mode.
Have you read this? https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails-build.html (configuring the host).
 

asteriskRoss

Well-Known Member

Reaction score: 153
Messages: 445

I can connect to it locally no problem, on the same machine, but I can't connect to my jail from the LAN... I need to open the jail to the LAN, and assigning a secondary IP to my interface doesn't work, and ideally, I'd want my jail to acquire a DHCP lease from the router...
you need to setup your network jail with VNET
If you want your jail to use IPv4 with DHCP then VNET will indeed work. A quick search on these forums and the web picked up some threads and tutorials that will point you in the right direction. An alternative would be to use private addresses for your jails and configure NAT on your host system, though with only one public IP address, you can only run one service per port number; for example, your host and jail couldn't both run a web server reachable on port 80 of your public IP address. That may or may not be a concern for you.

You mentioned you're using IPFW. Documentation for setting up NAT is hard to find, though it certainly supports NAT. I used PF for 5 years then IPFW for 5 years and recently bit the bullet to port my rules back to PF for new hosts. In my humble opinion, the PF rule syntax is clearer and easier to maintain and I'm yet to find something I was doing with IPFW that I can't do with anchors and and a custom rc(8) for adding extra rules after networking is fully up and DHCP leases have been acquired.
 

PMc

Aspiring Daemon

Reaction score: 327
Messages: 825

I suggest a layered approach, where the first step is to get plain basic connectivity. Things like vlan, ipfw etc. all build upon the basic connectivity and tend to make it more complicated, not resolve problems. And http features have nothing at all to do with it.

A. Traditional jails

These get an IP address, which is usually an alias on lo0[1]. The base machine then knows itself with that IP address and will accept packets for it. It is then your responsibility to make certain that only processes running in the jail do listen on that address.
Outgoing packets will go thru the routing of the base machine and be sent away as usual.
Obviousely the outside world should route the jail's address to the base machine.

Special case: routeable "real" Internet address on a jail within a RFC1918 LAN.
The jail is assigned this IP address just like above, and it is put onto lo0 like above. The base machine will now accept packets destined to this address, and again you must make sure that it is listened to only with processes running in the jail.
Then you need two things:
  1. some means to make the arriving packets appear in the base machine. E.g. routing them to it, making them appear on some tunnel or vpn, or whatever is appropriate. If a service must run to achieve this, that service must run on the base machine.
  2. some means to get the packets that are created by the jail sent out on the right way (they already have a "real" IP sender address and may need to bypass NAT, or be routed differently, etc.). This will require some source-based routing, and can be achieved with ipfw(8) forward.

B. VIMAGE (vnet) jails

These behave in all networking regards as a separate host. Not a separate host on the net, but a separate host, and it is up to you to create any kind of networking connections to wire them together.
The wiring is usually done by virtual entities (but you could probably as well insert physical interface cards and dedicate them to various vnet jails). I am using ng_eiface(4) and ng_bridge(4) entities, others prefer if_bridge(4).

So I suggest you start looking at it on this level, and plan on how to make it work (as a result, LAN connectivity should then work). Then in the next step consider the features required (firewall, VLAN, DHCP, whatever) and how they should integrate. I never worked with DHCP so I cannot advice on this, but I suppose it complicated with traditional jails, because they usually work on a fixed predefined IP. With VIMAGE it would be done like it were an additional machine standing there (probably wiring the jail to a virtual bridge which also connects the router).
For the other features (except DHCP) feel free to ask for further details.


[1] some people insist on using lo1 instead of lo0.


You mentioned you're using IPFW. Documentation for setting up NAT is hard to find
Yes, and much if it is erroneous if not outright bogus. So I have made code that creates the rules in an automated fashion (supporting an arbitrary number of NATs in any arrangement), and I think that's the way to go: the gory details should be automated, just like with compilers.
 
Top