Solved Jails and public IPv6 addresses

[janedenone]

For a couple of VNET jails, I used private subnets (IPv4 and IPv6) so far, and pf for NAT. Switching to public IPv6 addresses has proven surprisingly difficult. My setup looks like this:

• VPS with a /64 IPv6 subnet
• A bridge interface with an IP address from a /120 subnet ( 2a0a:4cc0:c1:5ba9::100/120)
• 4 VNET jails with interfaces linked to the same /120 subnet ( 2a0a:4cc0:c1:5ba9::101/120 etc)

As soon as I disable the pf NAT rule for IPv6, connectivity to any external address is lost for the jails.
After testing many things, I finally found the following advice (which leads to a working configuration):

sysctl net.inet6.icmp6.nd6_onlink_ns_rfc4861=1
ndp -s 2a0a:4cc0:c1:5ba9::110 58:9c:fc:10:1c:83 proxy

However, a Wiki entry by Lennert van Alboom advises against this, suggesting to assign the whole /64 subnet to the bridge instead, and to change the ipv6_defaultrouter from fe80::1%vtnet0 to fe80::1%bridge0. I am a little afraid this might break my access to the VPS, so I would like to understand the suggestion better:

• Why is it helpful to assign the /64 subnet (instead of a smaller subnet) to the bridge? How would you configure a setup with multiple subnets, each linked to a different bridge interface?
• Why is disabling TCP segmentation offloading for the physical interface important (according to Lennert)?
• What are the effects from setting the bridge interface as default router instead of the "physical" interface on the host (specifically: What happens to the current primary IPv6 address of the host when assigning the /64 subnet containing this IP to the bridge?)

 

[bvdw78]

I got a /64 from Hetzner. I cut a /66 from that and assigned it to bridge0. Jails have epairs that have an address in the /66 and the bridge has ::1 in the /66. The jail has the ::1 address as its default router. The host's igb0 interface has the ::1 from the overlying /64 and the default gateway is the link-local address of Hetzner's upstream router fe80::1%igb0. Did have to turn on IPv6 forwarding before this would work.

I had to do this with a bridge inside my box because Hetzner doesn't like seeing MAC's on L2 going out the server's network port. The bridge handles that, and all the rest is very simple L3 routing based on default routers all the way up.

Tip: I generally use IPv4 as a lifeline when working on IPv6 and vice versa.

 

[janedenone]

I got a /64 from Hetzner. I cut a /66 from that and assigned it to bridge0. Jails have epairs that have an address in the /66 and the bridge has ::1 in the /66. The jail has the ::1 address as its default router. The host's igb0 interface has the ::1 from the overlying /64 and the default gateway is the link-local address of Hetzner's upstream router fe80::1%igb0. Did have to turn on IPv6 forwarding before this would work.

That describes my initial setup exactly, but it required a NAT rule for IPv6 to get jails connectivity to the Internet (for outgoing traffic). I have been looking all day for the missing element. This is /etc/rc.conf on the host:

ipv6_activate_all_interfaces="YES"
ipv6_defaultrouter=fe80::1%vtnet0
ifconfig_vtnet0_ipv6="inet6 2a0a:4cc0:c1:5ba9:b43b:46ff:fe58:9660"

cloned_interfaces="bridge0"
ifconfig_bridge0_name="bastille0"
ifconfig_bastille0="addm vtnet0 up"
ifconfig_bastille0_ipv6="inet6 2a0a:4cc0:c1:5ba9::100/120"

/etc/sysctl.conf contains:

net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1

And this is a jail's /etc/rc.conf:

ifconfig_e0b_webjail_name="vnet0"
ifconfig_vnet0_ipv6="inet6 -ifdisabled 2a0a:4cc0:c1:5ba9::110/120"
ipv6_defaultrouter="2a0a:4cc0:c1:5ba9::100"

Apart from having :100 as the first address in my /120 subnet, the configuration is seemingly identical to what you describe, and yet, it does not work.

Tip: I generally use IPv4 as a lifeline when working on IPv6 and vice versa.

You're right – that is reassuring.

 

[bvdw78]

You're not setting a prefixlen on vtnet0. Here's the relevant bit of my box:

ifconfig_igb0="DHCP"
ifconfig_igb0_ipv6="inet6 2a01:4f9:6a:1955::1 prefixlen 65"
ipv6_defaultrouter="fe80::1%igb0"
firewall_enable="YES"
firewall_nat_enable="YES"
firewall_script="/mgt/bin/natbox.sh"
cloned_interfaces="bridge0"
ifconfig_bridge0="inet 10.20.0.1 netmask 255.255.255.0 up"
ifconfig_bridge0_ipv6="inet6 2a01:4f9:6a:1955:8000::1 prefixlen 66"
gateway_enable="YES"
ipv6_gateway_enable="YES"

The firewall NAT part is strictly IPv4 only. Nothing happens in natbox.sh that has any relevance for IPv6. I'm setting igb0 statically because of the /65 prefix. You do *NOT* want to overlap the prefix with something inside it. So I'm guessing your omission of the prefixlen is going to default to /64, which overlaps with the /120 you're using inside. In my case I chopped my /64 into 2 chunks initially, each /65. The first /65 is assigned to the host. The second /65 goes to the bridge. They never overlap.

There used to be another such split, hence the prefixlen 66 there that I never bothered to clean up back to /65. Conceptually your setup should work as long as no prefixes overlap. If they do, the kernel won't know where to route anything which is exactly what you're seeing.

 

[janedenone]

Thank you! I replicated the setup like this (leaving out the IPv4 parameters):

ipv6_activate_all_interfaces="YES"
ifconfig_vtnet0_ipv6="inet6 2a0a:4cc0:c1:5ba9::1 prefixlen 65"
ipv6_defaultrouter=fe80::1%vtnet0
ipv6_gateway_enable="YES"

cloned_interfaces="bridge0"
ifconfig_bridge0_name="bastille0"
ifconfig_bastille0="addm vtnet0 up"
ifconfig_bastille0_ipv6="inet6 2a0a:4cc0:c1:5ba9:8000::1 prefixlen 65"

The jails are configured accordingly:

ifconfig_e0b_webjail_name="vnet0"
ifconfig_vnet0_ipv6="inet6 -ifdisabled 2a0a:4cc0:c1:5ba9:8000::10 prefixlen 65"
ipv6_defaultrouter="2a0a:4cc0:c1:5ba9:8000::1"

Unfortunately, I can ping6 the vtnet0 gateway address from within the jails, but still nothing beyond the host.

 

[bvdw78]

You're adding your vtnet0 to the bridge. Don't do that. The OS routes between the bridge and your host.

 

[janedenone]

You're adding your vtnet0 to the bridge. Don't do that. The OS routes between the bridge and your host.

Changed that, but still no connectivity from the jails. ifconfig returns the following now:

root@host:~ # ifconfig vtnet0
vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=ec079b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS>
    ether b6:3b:46:58:96:60
    inet 159.195.29.106 netmask 0xfffffc00 broadcast 159.195.31.255
    inet6 fe80::b43b:46ff:fe58:9660%vtnet0 prefixlen 64 scopeid 0x1
    inet6 2a0a:4cc0:c1:5ba9::1 prefixlen 65
    media: Ethernet autoselect (10Gbase-T <full-duplex>)
    status: active
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

root@host:~ # ifconfig bastille0
bastille0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=10<VLAN_HWTAGGING>
    ether 58:9c:fc:10:39:48
    inet 10.95.0.1 netmask 0xffffff00 broadcast 10.95.0.255
    inet6 2a0a:4cc0:c1:5ba9:8000::1 prefixlen 65
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    bridge flags=0<>
    member: e0a_webjail flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            port 20 priority 128 path cost 2000 vlan protocol 802.1q

root@webjail:~ # ifconfig vnet0
vnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=60000b<RXCSUM,TXCSUM,VLAN_MTU,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 58:9c:fc:10:1c:83
    inet 10.95.0.10 netmask 0xffffff00 broadcast 10.95.0.255
    inet6 fe80::5a9c:fcff:fe10:1c83%vnet0 prefixlen 64 scopeid 0x15
    inet6 2a0a:4cc0:c1:5ba9:8000::10 prefixlen 65
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

 

[mumu]

but still no connectivity from the jails

What you could try next is running ndproxy on the host..

pkg install ndproxy-3.2.1403000_1
/etc/rc.conf:

# NDPROXY
ndproxy_enable="YES"
ndproxy_uplink_interface="vtnet0" # host interface
ndproxy_downlink_mac_address="b6:3b:46:58:96:60" # host mac address
ndproxy_uplink_ipv6_addresses="fe80::1" # gateway(s) address(es) or mac address separated by semi-colon

Start ndproxy: service ndproxy start

Then try to ping anything from the jail.

 

[bvdw78]

Your jails should have 2a0a:4cc0:c1:5ba9:8000::1 as their default gateway. On the host you must have sysctl net.inet6.ip6.forwarding: 1. Here's the relevant bits of a jail config from my end, snipped a lot of things that are required but irrelevant to the networking bits:

jailname {
  vnet;
  exec.prestart = "ifconfig epair9 create up";
  exec.prestart += "ifconfig epair9a up";
  exec.prestart += "ifconfig bridge0 addm epair9a up";
  vnet.interface = "epair9b";
  exec.start = "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.poststop = "ifconfig epair9a destroy";
}

The running jail's ifconfig:

epair9b: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=200009<RXCSUM,VLAN_MTU,RXCSUM_IPV6>
    ether 58:9c:fc:10:cc:cd
    inet 10.20.0.12 netmask 0xffffff00 broadcast 10.20.0.255
    inet6 fe80::5a9c:fcff:fe10:cccd%epair9b prefixlen 64 scopeid 0x6
    inet6 2a01:4f9:6a:1955:8000::c prefixlen 66
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Here's the bridge on my host (snipped a dozen other unrelated epairs):

bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=10<VLAN_HWTAGGING>
    ether 58:9c:fc:10:68:c6
    inet 10.20.0.1 netmask 0xffffff00 broadcast 10.20.0.255
    inet6 2a01:4f9:6a:1955:8000::1 prefixlen 66
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    bridge flags=0<>
    member: epair9a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            port 5 priority 128 path cost 2000 vlan protocol 802.1q
    groups: bridge vm-switch viid-a7d0a@
    nd6 options=1<PERFORMNUD>

I have two such setups running. These bits come from my prod environment. I have a Raspberry Pi in the closet here that's running the same setup. No need for ndproxy on my end, but your mileage may vary.

I'd be interested in what tcpdump tells you when you're looking at your bridge while pinging something on the internet from within a jail. It may be sending packets out but not handling the return traffic properly. All kinds of subtlety could still be wrong.

 

[janedenone]

Your jails should have 2a0a:4cc0:c1:5ba9:8000::1 as their default gateway.

root@webjail:~ # netstat -rn -6
Routing tables
Internet6:
Destination                       Gateway                       Flags         Netif Expire
::/96                             link#22                       URS             lo0
default                           2a0a:4cc0:c1:5ba9:8000::1     UGS           vnet0

On the host you must have sysctl net.inet6.ip6.forwarding: 1.

root@host:~ # sysctl net.inet6.ip6.forwarding
net.inet6.ip6.forwarding: 1

I'd be interested in what tcpdump tells you when you're looking at your bridge while pinging something on the internet from within a jail. It may be sending packets out but not handling the return traffic properly. All kinds of subtlety could still be wrong.

root@host:~ # tcpdump -i bastille0
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on bastille0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:39:54.725766 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 0, length 16
13:39:55.760707 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 1, length 16
13:39:56.810715 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 2, length 16
13:39:57.816938 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 3, length 16
13:39:58.860681 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 4, length 16
13:39:59.876961 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 5, length 16
13:40:00.396938 IP6 2a0a:4cc0:c1:5ba9:8000::10 > 2a0a:4cc0:c1:5ba9:8000::1: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::1, length 32
13:40:00.396989 IP6 2a0a:4cc0:c1:5ba9:8000::1 > 2a0a:4cc0:c1:5ba9:8000::10: ICMP6, neighbor advertisement, tgt is 2a0a:4cc0:c1:5ba9:8000::1, length 24
13:40:00.938182 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 6, length 16
13:40:01.960691 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 7, length 16
13:40:03.033208 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 20809, seq 8, length 16

 

[bvdw78]

That looks reasonable. Next step: does the host network interface see the pings going out? An output of 'netstat -rn' on the host and in a jail could also tell more about the route tables the system is using.

 

[janedenone]

This happens on the host's interface:

root@host:~ # tcpdump -i vtnet0 | grep 2a0a:4cc0:c1:5ba9:8000
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:14:28.828205 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 21219, seq 0, length 16
14:14:29.563753 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
14:14:29.860634 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 21219, seq 1, length 16
14:14:30.564414 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
14:14:30.933195 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 21219, seq 2, length 16
14:14:31.563527 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
14:14:32.005751 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 21219, seq 3, length 16
14:14:33.078262 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 21219, seq 4, length 16
14:14:33.566361 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
14:14:34.150788 IP6 2a0a:4cc0:c1:5ba9:8000::10 > one.one.one.one: ICMP6, echo request, id 21219, seq 5, length 16
14:14:34.567034 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32

In the jail, nothing is displayed, and the initial request only appears after I stop tcpdump:

root@webjail:~ # tcpdump -i vnet0
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C
^C14:23:07.580682 IP6 2a0a:4cc0:c1:5ba9:8000::10 > 2606:4700:4700::1111: ICMP6, echo request, id 21419, seq 0, length 16

1 packet captured
21 packets received by filter
0 packets dropped by kernel

 

[bvdw78]

How about the route tables on both the jail and the host? netstat -rn?

 

[janedenone]

root@host:~ # netstat -rn -6
Routing tables

Internet6:
Destination                       Gateway                       Flags         Netif Expire
::/96                             link#2                        URS             lo0
default                           fe80::1%vtnet0                UGS          vtnet0
::1                               link#2                        UHS             lo0
::ffff:0.0.0.0/96                 link#2                        URS             lo0
2a0a:4cc0:c1:5ba9::/65            link#1                        U            vtnet0
2a0a:4cc0:c1:5ba9::1              link#2                        UHS             lo0
2a0a:4cc0:c1:5ba9:8000::/65       link#3                        U         bastille0
2a0a:4cc0:c1:5ba9:8000::1         link#2                        UHS             lo0
fd00:1234:5678:abcd:ef01:2345:6789:0/120 link#4                 U               wg0
fd00:1234:5678:abcd:ef01:2345:6789:1 link#2                     UHS             lo0
fe80::%lo0/10                     link#2                        URS             lo0
fe80::%vtnet0/64                  link#1                        U            vtnet0
fe80::b43b:46ff:fe58:9660%lo0     link#2                        UHS             lo0
fe80::%lo0/64                     link#2                        U               lo0
fe80::1%lo0                       link#2                        UHS             lo0
ff02::/16                         link#2                        URS             lo0

root@webjail:~ # netstat -rn -6
Routing tables

Internet6:
Destination                       Gateway                       Flags         Netif Expire
::/96                             link#22                       URS             lo0
default                           2a0a:4cc0:c1:5ba9:8000::1     UGS           vnet0
::1                               link#22                       UHS             lo0
::ffff:0.0.0.0/96                 link#22                       URS             lo0
2a0a:4cc0:c1:5ba9:8000::/65       link#21                       U             vnet0
2a0a:4cc0:c1:5ba9:8000::10        link#22                       UHS             lo0
fe80::%lo0/10                     link#22                       URS             lo0
fe80::%vnet0/64                   link#21                       U             vnet0
fe80::5a9c:fcff:fe10:1c83%lo0     link#22                       UHS             lo0
fe80::%lo0/64                     link#22                       U               lo0
fe80::1%lo0                       link#22                       UHS             lo0
ff02::/16                         link#22                       URS             lo0

 

[bvdw78]

I don't know what you're doing firewall-wise, but it looks like things are actively being blocked on your end. I can ping your host from my side, but not your bridge or the jail inside it. I'll leave ping running for a while. That'll come into your system from 2a01:4f9:6a:1955:8000::4 a packet per second. Can you see those pings going into your bridge through to your jail at all?

Traceroutes from both my home ISP and my Hetzner server are strange indeed. Very long delays, alternate paths.. something is up with IPv6 at your hoster.

 

[bvdw78]

This may also be relevant for you:

P

Thread 'Netcup IPv6 Intermittant'

Jan 19, 2025

Hi everyone,
I am aware there are a number of threads about this both here on FreeBSD's forums and on netcup itself, but largely the end result is to use another provider. Which is fine, but I've just entered a 6 month lease of a VPS with Netcup so would be nice to have a solution.
For those unaware, it seems to be an issue when running FreeBSD (seemingly 13 and above judging by forum posts, I am running 14.2) on Netcup VPSs that IPv6 connectivity is very intermittant. My experience has been that IPv6 works for about 5 minutes after reboot, then drops off and stops working until next...

• pswilde
• ipv6 netcup network
• Replies: 8
• Forum: Networking

 

[janedenone]

I don't know what you're doing firewall-wise, but it looks like things are actively being blocked on your end. I can ping your host from my side, but not your bridge or the jail inside it. I'll leave ping running for a while. That'll come into your system from 2a01:4f9:6a:1955:8000::4 a packet per second. Can you see those pings going into your bridge through to your jail at all?

I disabled pf while testing. This is what I see:

root@host:~ # tcpdump -i vtnet0 | grep 2a0a:4cc0:c1:5ba9:8000::1
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:19:46.048311 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
15:19:58.054304 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
15:19:59.054500 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32
15:20:00.053660 IP6 fe80::be0f:fe08:b30:a843 > ff02::1:ff00:10: ICMP6, neighbor solicitation, who has 2a0a:4cc0:c1:5ba9:8000::10, length 32

No packets with your source IP on the network interface.

 

[janedenone]

This may also be relevant for you:

P

Thread 'Netcup IPv6 Intermittant'

Jan 19, 2025

Hi everyone,
I am aware there are a number of threads about this both here on FreeBSD's forums and on netcup itself, but largely the end result is to use another provider. Which is fine, but I've just entered a 6 month lease of a VPS with Netcup so would be nice to have a solution.
For those unaware, it seems to be an issue when running FreeBSD (seemingly 13 and above judging by forum posts, I am running 14.2) on Netcup VPSs that IPv6 connectivity is very intermittant. My experience has been that IPv6 works for about 5 minutes after reboot, then drops off and stops working until next...

• pswilde
• ipv6 netcup network
• Replies: 8
• Forum: Networking

Interesting. I did experience issues connecting to IPv6 hosts for a couple of minutes once, and connecting to the host via IPv6 usually takes 3-4 seconds (vs. instantly with IPv4). But IPv6 connectivity for the host itself works now.

 

[bvdw78]

It's really strange that you're not seeing any packets whatsoever on the host. The ISP must route the packages to your host interface because that's where its router should deliver anything bound for your /64. It shouldn't care what you do with your /64 behind that, but it sure starts to look like it *does* care. You should, at the very least, see my pings coming into your host network interface so your OS can act on them (forward them, drop them, whatever.. but at least show something). The fact that you see nothing arriving at all would be reason to ask the ISP.

A traceroute from my box at Hetzner doesn't look very good either:

# traceroute6 -n 2a0a:4cc0:c1:5ba9:8000::1
traceroute6 to 2a0a:4cc0:c1:5ba9:8000::1 (2a0a:4cc0:c1:5ba9:8000::1) from 2a01:4f9:6a:1955::1, 64 hops max, 28 byte packets
 1  2a01:4f9::a:6:a  0.304 ms  0.390 ms  0.377 ms
 2  2a01:4f8:0:3::3b9  0.300 ms  0.281 ms  0.238 ms
 3  2a01:4f8:0:3::2e9  20.108 ms
    2a01:4f8:0:3::2e5  20.089 ms  20.110 ms
 4  2a01:4f8:0:3::2fd  20.381 ms  20.349 ms
    2a01:4f8:0:3::34d  20.327 ms
 5  2001:7f8::8463:0:2  22.497 ms  22.545 ms  22.498 ms
 6  2a01:4a0:0:2003::2  24.939 ms
    2a01:4a0:0:2025::4  22.595 ms
    2a01:4a0:0:2003::2  24.859 ms
 7  2a01:4a0:0:2003::2  24.935 ms  24.943 ms  25.053 ms
 8  *
    2a01:4a0:0:2015::26  25.254 ms  25.227 ms
 9  * *
    2a01:4a0:1338:77::2  2050.995 ms
10  * * *
11  2a01:4a0:1338:77::2  2033.132 ms !A *  2032.342 ms !A

It doesn't even end up at your box at all, and a !A is very bad news.

 

[janedenone]

It's really strange that you're not seeing any packets whatsoever on the host. The ISP must route the packages to your host interface because that's where its router should deliver anything bound for your /64. It shouldn't care what you do with your /64 behind that, but it sure starts to look like it *does* care. You should, at the very least, see my pings coming into your host network interface so your OS can act on them (forward them, drop them, whatever.. but at least show something). The fact that you see nothing arriving at all would be reason to ask the ISP.

A traceroute from my box at Hetzner doesn't look very good either:

[snip]

It doesn't even end up at your box at all, and a !A is very bad news.

I just switched to a different static IPv6 address (2a0a:4cc0:c1:5ba9::2) for the host, and I can ping the host under this address (from a box at Hetzner), but traceroute shows this:

root@web:~# traceroute -n 2a0a:4cc0:c1:5ba9::2
traceroute to 2a0a:4cc0:c1:5ba9::2 (2a0a:4cc0:c1:5ba9::2), 30 hops max, 80 byte packets
 1  fe80::1%eth0  8.663 ms  8.984 ms  9.050 ms
 2  2a01:4f8:0:e172::432f  4.259 ms  4.350 ms  4.342 ms
 3  * * *
 4  2a01:4f8:cfff:2::f000:405  5.273 ms 2a01:4f8:cfff:2::f000:401  4.685 ms  5.255 ms
 5  2a01:4f8:0:e172::a1e1  4.178 ms 2a01:4f8:0:e172::a1d5  4.265 ms 2a01:4f8:0:e172::a1e1  4.256 ms
 6  2a01:4f8:0:3::5b2  4.247 ms 2a01:4f8:0:3::5c2  0.883 ms 2a01:4f8:0:3::5ca  0.845 ms
 7  2a01:4f8:0:3::3f1  3.464 ms  3.453 ms 2a01:4f8:0:3::60e  24.517 ms
 8  2a01:4a0:1338:16b::1  3.435 ms  3.427 ms  2.920 ms
 9  2a01:4a0:0:2014::2  3.429 ms  3.416 ms  3.401 ms
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

All very confusing. I will contact my ISP and post any progress here. Thank you very much for your help and patience!

 

[mumu]

Hm, is the /64 an on-link prefix or it is actually routed to your VM ? I am guessing it is an on-link /64 which has tons of issues with it related to *BSD.

 

[janedenone]

Hm, is the /64 an on-link prefix or it is actually routed to your VM ? I am guessing it is an on-link /64 which has tons of issues with it related to *BSD.

In the Netcup forums, someone said the IPv6 subnets were "switched". I will add this aspect to my support request.

 

[mumu]

In the Netcup forums, someone said the IPv6 subnets were "switched". I will add this aspect to my support request.

I have a Netcup VM that had this issue in the past. IPv6 is stable now. Here's my rc.conf:

ifconfig_vtnet0_ipv6="inet6 2a03:4000:4e:xxx::1/48"
#ipv6_defaultrouter="fe80::1%vtnet0"
ipv6_defaultrouter="2a03:4000:4e::2"

However, if the vnet+bridge still doesn't work, try using it with ndproxy on the host. If that doesn't work either, then maybe consider not using vnet+bridge and use ip6.addr instead in your jail.conf where the addresses are dynamically added/removed from vtnet0.

The defaultrouter address is taken from ndp -na, look at the Linklayer Address to find the gateway address.

 

[janedenone]

I created a small VPS at Hetzner to verify the difference between the two ISP's handling of IPv6 subnets, and can confirm what bvdw78 already stated on Friday: With a routed subnet at Hetzner, his setup with two non-overlapping /65 subnets works well. The Netcup server is just a test server, so I can easily migrate to Hetzner.

Thanks again for your help debugging this issue.