Solved cannot install in a jail with ports & jail defined with different subnets - does this work?

Hello,
I'm new to work with jail. I try this example:
FreeBSD jails: a complete example
I've created the jail from another example - copied some files from the host - did a bit configuration - and it works: I can start it and connect in it with 'tcsh' command.
With the example above I try to establish an internet connection to the jail and like to install a webserver from the ports. But the fetch command hangs.
What is configured from the example above:
in host:rc.conf
Code:
# NETZWERK Virtual IP's for Jails
cloned_interfaces="lo1"
ifconfig_lo1_aliases="inet 172.17.2.1/32 \
                      inet 172.17.2.2/32"

# for jails with NAT
firewall_nat_enable="YES"
gateway_enable=YES

jail_devfs_enable="YES"
jail_devfs_ruleset="devfsrules_for_jails"    #from /etc/devs.rules

in host:/etc/jails.conf
Code:
...

    ip4.addr = 172.17.2.1/24;
In the example he's using a pf firewall, but I use ipfw - so I search around and find that downstairs, what should be the same like it was described in pf
in host:/etc/ipfw.rule
Code:
cmd="ipfw -q add"
$cmd 00410 nat 1 ip from any to me in via $nwinterface
#
# Example told: "I’ve enabled NAT for an entire /29, which gives us 14 IP addresses to play with (172.17.2.1-14)."

$cmd 00420 nat 1 ip from 172.17.2.0/29 to any out via $nwinterface
#
ipfw nat 1 config if $nwinterface redirect_port tcp 172.17.2.1:80 80
ipfw nat 2 config if $nwinterface redirect_port tcp 172.17.2.1:443 443
his example for pf:
Code:
if = "igb0"
extip = "198.51.100.156"
jailnet = "172.17.2.0/29"
nat pass on $if from $jailnet to any -> $extip

#and

jailwww = "172.17.2.1"
wwwports = "{ 80 443 }"
rdr pass on $if proto tcp from any to $extip port $wwwports -> $jailwww
pass quick from $jailwww to $jailwww
First: Are my translation from pf to ipfw rules ok, or is still there something missing?

Can this example work with the different netmask's /32 /29 and /24 ?
It's been told that computer without a router can only reach an other within the same netmask range.

In the jail, jail:sockstat or jail:netstat shows nothing, but sure yet there is no service running. ifconfig on the host shows the both defined IPs 172.17.2.1 172.17.2.2 within the interface lo1, so they are there.

Installing some, inside the jail from the ports (ports tree is nullfs mounted from the host, as discribed in the example) the command make build hangs, because 'fetch' hangs - it cannot fetch anything. So the network configuration for the jail is not working.

Can anyone give help?
 
Last edited by a moderator:
First of all check your lo1 IP-alias configuration.
Read the FreeBSD handbook chapter '12.6. Virtual Hosts'.
The calculation of alias netmasks is important. For a given interface, there must be one address which correctly represents the network’s netmask. Any other addresses which fall within this network must have a netmask of all 1s, expressed as either 255.255.255.255 or 0xffffffff.

Example of correct lo1 aliases:
Code:
cloned_interfaces="lo1"
ifconfig_lo1="inet 192.168.46.1  netmask 255.255.255.0"
ifconfig_lo1_alias0="inet 192.168.46.2  netmask 255.255.255.255"
ifconfig_lo1_alias1="inet 192.168.46.3  netmask 255.255.255.255"

You can use jail networking with any correct mask like /23 etc.

It is correct to use 'ipfw nat' if you want to have internet and port-forwarding inside the jail on loopback interface.
But I prefer to use a reverse-proxy like nginx on the host instead of forwarding tcp 80,443 inside the jail.
Also you can use nginx-streams as a tcp-proxy for jail's ssh.

What about your NAT example:
Nat requre NATing of reverse traffic. For example:
nat 1 ip from 172.17.2.0/29 to any via ${IFWAN}
nat 1 ip from any to ${WAN_IP} via ${IFWAN}

Your NAT rules must cover the outgoing and incoming traffic that forwarded via NAT port forwarding.
 
Follow you advice I would translate my configuration with CIDR in
Code:
cloned_interfaces="lo1"

ifconfig_lo1="inet 172.17.2.0/24
ifconfig_lo1_alias0="inet 172.17.2.1/32
ifconfig_lo1_alias1="inet 172.17.2.2/32
Is this now correct?

Can I mix in rc.conf IP configurations with CIDR and some with "netmask x.x.x.x"?

I will also read 12.6 Virtual hosts.
 
I have tried you recommended settings in /etc/rc.conf and it knocks me out to single user mode with /etc read only:
Code:
ifconfig_lo1="inet 172.17.2.0/24
ifconfig_lo1_alias0="inet 172.17.2.1/32
ifconfig_lo1_alias1="inet 172.17.2.2/32
maybe because the internet interface has also /24 settings.

My settings gives me:
Code:
in rc.conf
# NETZWERK Virtual IP's for Jails
cloned_interfaces="lo1"
ifconfig_lo1_aliases="inet 172.17.2.1/32 \
                     inet 172.17.2.2/32"

#ifconfig
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 172.17.2.1 netmask 0xffffffff
        inet 172.17.2.2 netmask 0xffffffff
        inet6 fe80::1%lo1 prefixlen 64 scopeid 0x3
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

so it seems, that the settings in /etc/rc.conf are not the problem, that from the jail fetch does not connect to anything.
 
Please show the final configuration: /etc/rc.conf /etc/jail.conf and ipfw rules.

Can I mix in rc.conf IP configurations with CIDR and some with "netmask x.x.x.x"?
I do not know, it is a subject for another research.

maybe because the internet interface has also /24 settings.
You can use any correct mask for LAN and WAN interface.
Usually your internet service provider gives you all required IP configuration (via DHCP or on the sheet of paper)
You must have different networks on the both interfaces of your router.
For example: WAN 192.168.10.0/24 and LAN 192.168.20.0/24 is a correct configuration.

that from the jail fetch does not connect to anything
Just review your NAT and DNS configuration.
 
I have few servers with the similar configuration.
There is a simplified example of my configs.
/etc/rc.conf
Code:
ifconfig_vtnet0="inet 1.1.9.4 netmask 255.255.255.0"
defaultrouter="1.1.9.1"

cloned_interfaces="lo1"
ifconfig_lo1="inet 192.168.46.1  netmask 255.255.255.0"
ifconfig_lo1_alias0="inet 192.168.46.2  netmask 255.255.255.255"
ifconfig_lo1_alias1="inet 192.168.46.3  netmask 255.255.255.255"

firewall_enable="YES"
firewall_script="/root/bin/rc.firewall"
firewall_nat_enable="YES"

gateway_enable="YES"

/etc/jail.conf
Code:
...
dbmorm64 {
        host.hostname = "dbmorm64.x.com";
        path = "/home/jails/dbmorm64";
        ip4.addr += "192.168.46.2/32";
        allow.raw_sockets = 0;
        mount.fstab = "/etc/fstab.dbmorm64";
        exec.consolelog = "/var/log/jail_dbmorm64_console.log";
}

/root/bin/rc.firewall
Code:
#!/bin/sh
ipfw='/sbin/ipfw -q'
if_wan='vtnet0'
if_wan_ip='1.1.9.4'
${ipfw} -f flush
...
${ipfw} nat 1 config ip 1.1.9.4 deny_in same_ports redirect_port tcp 192.168.46.3:2204 2204
...
${ipfw} add nat 1 tcp from any to any via vtnet0
${ipfw} add nat 1 udp from any to 8.8.8.8,8.8.4.4 via vtnet0
${ipfw} add nat 1 udp from 8.8.8.8,8.8.4.4 to me via vtnet0

Proof of work:
Code:
root@srv1.x.com:~ # jls
   JID  IP Address      Hostname                      Path
   101  192.168.46.2    dbmorm64.x.com      /home/jails/dbmorm64
   102  192.168.46.4    x.com               /home/jails/morm64

root@srv1.x.com:~ # jexec 102 csh

root@x:/ # cd /tmp/

root@x:/tmp # fetch --no-verify-peer "https://secure.eicar.org/eicar.com.txt"
eicar.com.txt                                           68  B  558 kBps    00s

root@x:/tmp # ll eicar.com.txt
-rw-r--r--  1 root  wheel  68  1 июля   2020 eicar.com.txt
 
With this settings I have fixed it.
in /etc/rc.conf
Code:
# NETZWERK Virtual IP's for Jails
cloned_interfaces="lo1"
ifconfig_lo1_aliases="inet 172.17.2.1/32 \
                      inet 172.17.2.2/32"

firewall_nat_enable="YES"
gateway_enable=YES

in /etc/jail.conf
Code:
ip4.addr = 172.17.1.1/32;
in /etc/ipfw.rules
Code:
cmd="ipfw -q add"
nwinterface="vtnet0"

${cmd} 03410 nat 1 ip from any to me in via $nwinterface
${cmd} 03420 nat 1 ip from 172.17.2.0/27 to any out via $nwinterface

ipfw nat 1 config if $nwinterface redirect_port tcp 172.17.2.1:80 80
ipfw nat 1 config if $nwinterface redirect_port tcp 172.17.2.1:443 443
But I don't now what was wrong.
 
Back
Top