what are you trying to accomplish with this?
root@W520:~ # iocage start 8.0
No default gateway found for ipv6.
* Starting 8_0
+ Started OK
+ Using devfs_ruleset: 1005 (iocage generated default)
+ Configuring VNET FAILED
ifconfig: interface epair0b does not exist
Stopped 8_0 due to VNET failure
Why should there be?I would be very surprised if there is no jail.conf type file for IOCage.
I don't really know how things have got into this state:-we statically configure our jails with jail.conf and have all the epairs configured inrc.conf, and have no trouble. idk how iocage complicates things, but this config works for us
Code:cloned_interfaces="bridge1 bridge69 bridge666 epair1 epair2 epair4 epair51 bge0.1 bge0.69 bge0.666"
em0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=e520bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS>
ether f0:de:f1:a4:d9:10
inet 192.168.1.21 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=0
ether 24:77:03:26:51:84
groups: wlan
ssid "" channel 1 (2412 MHz 11b)
regdomain FCC country US authmode OPEN privacy OFF txpower 30
bmiss 10 scanvalid 60 wme bintval 0
parent interface: iwn0
media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=10<VLAN_HWTAGGING>
ether 58:9c:fc:10:16:f1
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
bridge flags=0<>
member: epair3a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 14 priority 128 path cost 2000 vlan protocol 802.1q
member: vnet0.10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 10 priority 128 path cost 2000 vlan protocol 802.1q
member: vnet0.8 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 7 priority 128 path cost 2000 vlan protocol 802.1q
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 1 priority 128 path cost 20000 vlan protocol 802.1q
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 02:80:37:ec:02:00
media: Ethernet autoselect
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair2a: flags=1008842<BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
ether 58:9c:fc:10:02:2f
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair2b: flags=1008842<BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
ether 58:9c:fc:10:67:8b
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vnet0.8: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: associated with jail: test as nic: epair0b
options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
ether f2:de:f1:ae:1b:75
hwaddr 58:9c:fc:10:d1:76
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vnet0.10: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: associated with jail: bank as nic: epair0b
options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
ether f2:de:f1:12:52:b3
hwaddr 58:9c:fc:10:74:97
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair3a: flags=1008942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
ether 58:9c:fc:10:23:8d
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair3b: flags=1008842<BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
ether 58:9c:fc:10:1b:f4
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
After a reboot I get this:-have you mucked with the interfaces on your own? or did iocage do this after some command?
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=4e524bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether f0:de:f1:a4:d9:10
inet 192.168.1.21 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=0
ether 24:77:03:26:51:84
groups: wlan
ssid "" channel 1 (2412 MHz 11b)
regdomain FCC country US authmode OPEN privacy OFF txpower 30
bmiss 10 scanvalid 60 wme bintval 0
parent interface: iwn0
media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 02:80:37:ec:02:00
media: Ethernet autoselect
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=10<VLAN_HWTAGGING>
ether 58:9c:fc:10:16:f1
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
bridge flags=0<>
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
iocage in a VM and it works without issue.root@fbsd15:~ # iocage create -n j2 -r 15.0-RELEASE
j2 successfully created!
root@fbsd15:~ #
root@fbsd15:~ # iocage set vnet=on ip4_addr="vnet0|192.168.1.234/24" j2
vnet: 0 -> 1
ip4_addr: none -> vnet0|192.168.1.234/24
root@fbsd15:~ #
root@fbsd15:~ # iocage start j2
No default gateway found for ipv6.
* Starting j2
+ Started OK
+ Using devfs_ruleset: 1000 (iocage generated default)
+ Configuring VNET OK
+ Using IP options: vnet
+ Starting services OK
+ Executing poststart OK
root@fbsd15:~ #
root@fbsd15:~ # ifconfig bridge0
bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=10<VLAN_HWTAGGING>
ether 58:9c:fc:10:26:ba
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
bridge flags=0<>
member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 5 priority 128 path cost 2000 vlan protocol 802.1q
member: vtnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 1 priority 128 path cost 2000 vlan protocol 802.1q
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
root@fbsd15:~ #
root@fbsd15:~ #
root@fbsd15:~ # ifconfig vnet0.2
vnet0.2: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: associated with jail: j2 as nic: epair0b
options=200009<RXCSUM,VLAN_MTU,RXCSUM_IPV6>
ether 0a:74:c0:65:58:8b
hwaddr 58:9c:fc:10:80:e8
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@fbsd15:~ #
root@fbsd15:~ # iocage console j2
FreeBSD 15.0-RELEASE-p1 (GENERIC) releng/15.0-n280999-7bceec30b351
Welcome to FreeBSD!
Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories: https://www.FreeBSD.org/security/
FreeBSD Handbook: https://www.FreeBSD.org/handbook/
FreeBSD FAQ: https://www.FreeBSD.org/faq/
Questions List: https://www.FreeBSD.org/lists/questions/
FreeBSD Forums: https://forums.FreeBSD.org/
Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with: pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.
Show the version of FreeBSD installed: freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages: man man
FreeBSD directory layout: man hier
To change this login announcement, see motd(5).
root@j2:~ #
root@j2:~ #
root@j2:~ # host freebsd.org
freebsd.org has address 96.47.72.84
freebsd.org has IPv6 address 2610:1c1:1:606c::50:15
freebsd.org mail is handled by 30 mx66.freebsd.org.
freebsd.org mail is handled by 10 mx1.freebsd.org.
root@j2:~ #
iocage list:-+------+-------------+-------+--------------+------+
| JID | NAME | STATE | RELEASE | IP4 |
+======+=============+=======+==============+======+
| None | 143 | down | 13.5-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | 6.0 | down | 13.5-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | 8.0 | down | 13.5-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | FreeBSD-1.0 | down | 15.0-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | bank | down | 15.0-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | mailman | down | 15.0-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | mfsbsd | down | 15.0-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
| None | test | down | 15.0-RELEASE | DHCP |
+------+-------------+-------+--------------+------+
iocage create -n xxx -r latest vnet=on dhcp=on dhcp requires bpf, enabling!
xxx successfully created!
iocage start xxxNo default gateway found for ipv6.
* Starting xxx
+ Started OK
+ Using devfs_ruleset: 1000 (iocage generated default)
+ Configuring VNET OK
+ Using IP options: vnet
+ Starting services OK
+ Executing poststart OK
+ Acquiring DHCP address: FAILED, address received: ERROR, check jail logs
Stopped xxx due to DHCP failure
vnet0 instead of vtnet0 and iocage didn't even complain(probably because this network interface doesn't exist) , started the jail like nothing happened. vtnet0 make iocage scream, it really doesn't like that. iocage create -n jail_name -r 15.0-RELEASE
iocage set vnet=on ip4_addr="xx.xx.xx.xx/xx" ip6_addr=none defaultrouter="xx.xx.xx.xx" defaultrouter6=none jail_name
These are two different things, I am not an expert but this is how I see both:I have never set defaultrouter as my dhcp server handles that.
You also needDid you set the variable "defaultrouter" in the host /etc/rc.conf, VNET jails need this.
gateway_enable="YES" , or just put this net.inet.ip.forwarding=1 in /etc/sysctl.conf, both accomplish the same thing. Without this VNET jails won't work.There are couple other things to do, but all I am telling you is already written into the documentation.
CONFIG_VERSION:33
allow_chflags:0
allow_mlock:0
allow_mount:0
allow_mount_devfs:0
allow_mount_fdescfs:0
allow_mount_fusefs:0
allow_mount_linprocfs:0
allow_mount_linsysfs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_nfsd:0
allow_quotas:0
allow_raw_sockets:0
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
allow_tun:0
allow_vmm:0
assign_localhost:0
available:readonly
basejail:0
boot:0
bpf:1
children_max:0
cloned_release:15.0-RELEASE
comment:none
compression:on
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:auto
defaultrouter6:auto
depends:none
devfs_ruleset:4
dhcp:1
enforce_statfs:2
exec_clean:1
exec_created:/usr/bin/true
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:xxx
host_hostuuid:xxx
host_time:1
hostid:7ead3701-5147-11cb-8f5c-e31f104dc22c
hostid_strict_check:0
interfaces:vnet0:bridge0
ip4:new
ip4_addr:none
ip4_saddrsel:1
ip6:new
ip6_addr:none
ip6_saddrsel:1
ip_hostname:0
jail_zfs:0
jail_zfs_dataset:iocage/jails/xxx/data
jail_zfs_mountpoint:none
last_started:none
localhost_ip:none
login_flags:-f root
mac_prefix:f2def1
maxproc:off
memorylocked:off
memoryuse:off
min_dyn_devfs_ruleset:1000
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:0
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nat:0
nat_backend:ipfw
nat_forwards:none
nat_interface:none
nat_prefix:172.16
nmsgq:off
notes:none
nsem:off
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
plugin_name:none
plugin_repository:none
priority:99
pseudoterminals:off
quota:none
readbps:off
readiops:off
release:15.0-RELEASE
reservation:none
resolver:/etc/resolv.conf
rlimits:off
rtsold:0
securelevel:2
shmsize:off
stacksize:off
state:down
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:0
type:jail
used:readonly
vmemoryuse:off
vnet:1
vnet0_mac:f2def186b9e1 f2def186b9e2
vnet0_mtu:auto
vnet1_mac:none
vnet1_mtu:auto
vnet2_mac:none
vnet2_mtu:auto
vnet3_mac:none
vnet3_mtu:auto
vnet_default_interface:none
vnet_default_mtu:1500
vnet_interfaces:none
wallclock:off
writebps:off
writeiops:off
Jan 27 11:09:22 home kernel: lo0: link state changed to UP
Jan 27 11:09:22 home kernel: em0: link state changed to DOWN
Jan 27 11:09:22 home kernel: em0: link state changed to UP
Jan 27 11:52:26 home kernel: epair0a: Ethernet address: 58:9c:fc:10:42:4c
Jan 27 11:52:26 home kernel: epair0b: Ethernet address: 58:9c:fc:10:70:3e
Jan 27 11:52:26 home kernel: epair0a: link state changed to UP
Jan 27 11:52:26 home kernel: epair0b: link state changed to UP
Jan 27 11:52:26 home kernel: epair0a: changing name to 'vnet0.1'
Jan 27 11:52:26 home kernel: bridge0: link state changed to UP
Jan 27 11:52:26 home kernel: vnet0.1: promiscuous mode enabled
Jan 27 11:52:27 home kernel: lo0: link state changed to UP
Jan 27 11:53:59 home kernel: vnet0.1: link state changed to DOWN
Jan 27 11:53:59 home kernel: epair0b: link state changed to DOWN
Jan 27 11:53:59 home kernel: bridge0: link state changed to DOWN
Jan 27 13:01:17 home kernel: epair0a: Ethernet address: 58:9c:fc:10:42:4c
Jan 27 13:01:17 home kernel: epair0b: Ethernet address: 58:9c:fc:10:70:3e
Jan 27 13:01:17 home kernel: epair0a: link state changed to UP
Jan 27 13:01:17 home kernel: epair0b: link state changed to UP
Jan 27 13:01:17 home kernel: epair0a: changing name to 'vnet0.2'
Jan 27 13:01:17 home kernel: bridge0: link state changed to UP
Jan 27 13:01:17 home kernel: vnet0.2: promiscuous mode enabled
Jan 27 13:01:17 home kernel: lo0: link state changed to UP
Jan 27 13:02:49 home kernel: vnet0.2: link state changed to DOWN
Jan 27 13:02:49 home kernel: epair0b: link state changed to DOWN
Jan 27 13:02:49 home kernel: bridge0: link state changed to DOWN
iocage create -n jail_name -r 15.0-RELEASE
iocage set vnet=on dhcp=on bpf=on ip6_addr=none defaultrouter="xx.xx.xx.xx" defaultrouter6=none jail_name
iocage create -n xyz -r latestxyz successfully created!
iocage set vnet=on dhcp=on bpf=on ip6_addr=none defaultrouter="192.168.1.1" defaultrouter6=none xyzvnet: 0 -> 1
dhcp: 0 -> 1
bpf: 0 -> 1
ip6_addr: none -> none
defaultrouter: auto -> 192.168.1.1
defaultrouter6: auto -> none
iocage start xyz* Starting xyz
+ Started OK
+ Using devfs_ruleset: 1000 (iocage generated default)
+ Configuring VNET OK
+ Using IP options: vnet
+ Starting services OK
+ Executing poststart OK
+ Acquiring DHCP address: FAILED, address received: ERROR, check jail logs
Stopped xyz due to DHCP failure
May be, but VNET jails are a bit different than classic ones.I never touched/etc/sysctl.conf in the past when it worked
Before doing that you really should try to apply the recommended settings:Because of a DHCP failure I'm continuing my debugging of the problem in a new thread:-
net.inet.ip.forwarding=1 # Enable IP forwarding between interfacesnet.link.bridge.pfil_onlyip=0 # Only pass IP packets when pfil is enablednet.link.bridge.pfil_bridge=0 # Packet filter on the bridge interfacenet.link.bridge.pfil_member=0 # Packet filter on the member interfacenet.link.bridge.pfil_bridge=0 because with it, packets are allowed to go through the bridge without being filtered by the firewall which could be why the DHCP requests can't reach your VNET jails (that's my guess only).The thing is that I never set that before and it worked.Next time, start a jail with the right settings applied, and if any problem appears, then post the relevant part from iocage log.
Did it work with VNET jail? and same network configuration?The thing is that I never set that before and it worked.