Can't Get ipv6 Working (reliably) On Contabo VPS Freebsd 13.1

PatchyFog · Jan 1, 2023

Happy New Year, all.

I've spun up a Freebsd 13.1 VPS at Contabo using their image and then I freebsd-update-ed it to p3. They give me a /64 6 subnet and I'd like to statically allocate 2 of my 18 quintillion addresses for mail and other services. (For this, I have chosen nodes 5 and 23 in honor of Discordia and Max Headroom, respectively.) But even though my ifconfig seemed to be coming out OK, I could not reach any other hosts. Anyway, I was in the middle of writing a long, detailed screed here to ask for help when suddenly everything started working! I mean, after 2 days of NOTHING, and I really hadn't changed anything. (Contabo support has (so far) provided nothing but cut-and-paste non-responsive responses.)

At this point, I think their gateway router at link-local fe80::1 is not up consistently or I'm having trouble with discovery. When everythinig works ndp -a shows the router present at link addr 00:c0:1d:c0:ff:ee (top of list):

Code:

Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::1%eth0                         00:c0:1d:c0:ff:ee   eth0 23h59m5s  S R
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R

But when it doesn't, fe80::1 is missing and it's just my 2 static globals and my 1 auto link-local showing:

Code:

Neighbor                             Linklayer Address  Netif Expire    S Flags
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R

If I run ndp -a in a loop with a 2 second sleep when it is not working, sometimes I can catch it in some intermediate state:

Code:

Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::1%eth0                         (incomplete)        eth0 expired   I  1
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R

I would love to blame their gateway, but when it is broken I have noticed that just doing this:

Code:

ping6 -v fe80::1%eth0
PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0
32 bytes from fe80::1%eth0: Neighbor Advertisement
32 bytes from fe80::1%eth0: Neighbor Advertisement
^C
--- fe80::1%eth0 ping6 statistics ---
11 packets transmitted, 0 packets received, 100.0% packet loss

...usually causes the router to magically show up in ndp -aand everything starts working! (I mean, I've never gotten a ping response from it, even when things are working:

Code:

[shn ~] -> ping6 www.kame.net
PING6(56=40+8+8 bytes) 2605:a140:2114:2385::23 --> 2001:2f0:0:8800:226:2dff:fe0b:4311
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=0 hlim=47 time=164.888 ms
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=1 hlim=47 time=164.951 ms
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=2 hlim=47 time=164.620 ms
^C
--- mango.itojun.org ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 164.620/164.820/164.951/0.143 ms
[shn ~] -> ping6 fe80::1%eth0
PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0
^C
--- fe80::1%eth0 ping6 statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss

...but seeing the discovery packets come in with the -v switch to ping seems to indicate/change something. (Tells the router my host exists?)

So...is the problem on their end or mine? Is running ndp -a just reporting or does it change the system state? Does the ping6 -v "fix" give any indication what might be going on?

Requisite configs below. They don't change whether I have global connectivity or not.

Thanks all.

-- Ward

My rc.conf for this static configuration looks like this:

Code:

hostname=vmi1142385.contaboserver.net
ifconfig_DEFAULT="DHCP inet6 accept_rtadv"
growfs_enable="YES"
cloudinit_enable="YES"
# ------------------------------------------------------------------------------
ifconfig_vtnet0_name=eth0
ipv6_enabled="YES"
firewall_enable="NO"

defaultrouter=207.244.224.1
ifconfig_eth0="inet 207.244.238.92 netmask 255.255.240.0"
ifconfig_eth0_alias0="inet 207.244.238.92 netmask 255.255.240.0"
ifconfig_eth0_alias1="inet 144.126.129.136 netmask 255.255.240.0"


ipv6_defaultrouter="fe80::1%eth0"
ifconfig_eth0_ipv6="inet6 2605:a140:2114:2385::23 prefixlen 64"
ifconfig_eth0_alias2="inet6 2605:a140:2114:2385::23 prefixlen 64"
ifconfig_eth0_alias3="inet6 2605:a140:2114:2385::05 prefixlen 64"

and produces this interface:

Code:

[/etc] -> ifconfig eth0

eth0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
         ether 00:50:56:49:64:a7
        inet6 fe80::250:56ff:fe49:64a7%eth0 prefixlen 64 scopeid 0x1
        inet6 2605:a140:2114:2385::23 prefixlen 64
        inet6 2605:a140:2114:2385::5 prefixlen 64
        inet 207.244.238.92 netmask 0xfffff000 broadcast 207.244.239.255
        inet 144.126.129.136 netmask 0xfffff000 broadcast 144.126.143.255
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

And the routes:

Code:

Internet6:

Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::1%eth0                  UGS        eth0
::1                               link#2                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2605:a140:2114:2385::/64          link#1                        U          eth0
2605:a140:2114:2385::5            link#1                        UHS         lo0
2605:a140:2114:2385::23           link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%eth0/64                    link#1                        U          eth0
fe80::250:56ff:fe49:64a7%eth0     link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0

facedebouc · Jan 1, 2023

I just commented out all Contabo pre-filled stuff from /etc/rc.conf and statically configured my IPv6 network interface vtnet0.

Code:

ipv6_defaultrouter="fe80::1%vtnet0"
ifconfig_vtnet0_ipv6="inet6 2a02:c206:xxxx:xxxx::1 prefixlen 64"
rtsold_enable="YES"
rtsold_flags="-Fa"

As you can see I am starting the router solicitation daemon at boot.

PatchyFog · Jan 1, 2023

facedebouc said:
I just commented out all Contabo pre-filled stuff from /etc/rc.conf and statically configured my IPv6 network interface vtnet0.

Code:

ipv6_defaultrouter="fe80::1%vtnet0" ifconfig_vtnet0_ipv6="inet6 2a02:c206:xxxx:xxxx::1 prefixlen 64" rtsold_enable="YES" rtsold_flags="-Fa"

As you can see I am starting the router solicitation daemon at boot.

Ah, okay. I see you dropped the eth0 alias for vtnet0 -- but is rtsold the secret sauce? My thinking was that my host only needed to handle router solicitation messages (and rtsold) if I was doing SLAAC and being all dynamic and shit. No? I need it even with "defaultrouter" manually set? (BTW, my host has been up for hours since I did the ping -v thing and kicked fe80::1 into life.)

As you can see, I'm no expert. Thanks for your reply.

facedebouc · Jan 1, 2023

PatchyFog said:
Ah, okay. I see you dropped the eth0 alias for vtnet0 -- but is rtsold the secret sauce? My thinking was that my host only needed to handle router solicitation messages (and rtsold) if I was doing SLAAC and being all dynamic and shit. No? I need it even with "defaultrouter" manually set? (BTW, my host has been up for hours since I did the ping -v thing and kicked fe80::1 into life.)

As you can see, I'm no expert. Thanks for your reply.

That's the way it works for me.
Default configuration from Contabo with cloud-init does not work for me so I made some cleaning after installation of their FreeBSD 13.1 image.
I removed cloud-init configuration and package as well as Linuxisms like eth0 and /etc/timezone.
I statically assign IPv6 addresses to the host and his jails.
And yes rtsold(8) does the job.

facedebouc · Jan 1, 2023

And I also find that their sshd(8) and security/sudo configuration is too permissive so I changed that.

PatchyFog · Jan 1, 2023

Well, firing up rtsold at boot didn't change anything for me (nor did enabling ifconfig_eth0_ipv6="inet6 accept_rtadv" -- but I still kinda think I should not need either one since I am forcing it with defautrouter. But a quick ping6 fe80::1%eth0 makes the router pop right up as a neighbor in the ndp -a list and everything sings and dances after that. (So I mean, I could script that as a boot-time workaround if it comes to that, but would sure like to not have to.)

I followed your lead and dropped all their other cruft (and I agree with you about sshd and sudo -- I have my own config for the former I drag around.

Anyway, thanks -- it was worth a shot.

facedebouc · Jan 2, 2023

Don't need to ping6, I get the same result as yours after reboot with ndp -an, but after a first IPv6 request it's OK, the defaultrouter appears in the NDP table.
The first IPv6 DNS lookup does the job :
host -v -6 www.freebsd.org
I didn't see a loss of IPv6 connection after that.

mumu · Jan 21, 2023

PatchyFog said:
Happy New Year, all.

I've spun up a Freebsd 13.1 VPS at Contabo using their image and then I freebsd-update-ed it to p3. They give me a /64 6 subnet and I'd like to statically allocate 2 of my 18 quintillion addresses for mail and other services. (For this, I have chosen nodes 5 and 23 in honor of Discordia and Max Headroom, respectively.) But even though my ifconfig seemed to be coming out OK, I could not reach any other hosts. Anyway, I was in the middle of writing a long, detailed screed here to ask for help when suddenly everything started working! I mean, after 2 days of NOTHING, and I really hadn't changed anything. (Contabo support has (so far) provided nothing but cut-and-paste non-responsive responses.)

At this point, I think their gateway router at link-local fe80::1 is not up consistently or I'm having trouble with discovery. When everythinig works ndp -a shows the router present at link addr 00:c0:1d:c0:ff:ee (top of list):

Code:

Neighbor Linklayer Address Netif Expire S Flags fe80::1%eth0 00:c0:1d:c0:ff:ee eth0 23h59m5s S R 2605:a140:2114:2385::23 00:50:56:49:64:a7 eth0 permanent R 2605:a140:2114:2385::5 00:50:56:49:64:a7 eth0 permanent R fe80::250:56ff:fe49:64a7%eth0 00:50:56:49:64:a7 eth0 permanent R

But when it doesn't, fe80::1 is missing and it's just my 2 static globals and my 1 auto link-local showing:

Code:

Neighbor Linklayer Address Netif Expire S Flags 2605:a140:2114:2385::23 00:50:56:49:64:a7 eth0 permanent R 2605:a140:2114:2385::5 00:50:56:49:64:a7 eth0 permanent R fe80::250:56ff:fe49:64a7%eth0 00:50:56:49:64:a7 eth0 permanent R

If I run ndp -a in a loop with a 2 second sleep when it is not working, sometimes I can catch it in some intermediate state:

Code:

Neighbor Linklayer Address Netif Expire S Flags fe80::1%eth0 (incomplete) eth0 expired I 1 2605:a140:2114:2385::23 00:50:56:49:64:a7 eth0 permanent R 2605:a140:2114:2385::5 00:50:56:49:64:a7 eth0 permanent R fe80::250:56ff:fe49:64a7%eth0 00:50:56:49:64:a7 eth0 permanent R

I would love to blame their gateway, but when it is broken I have noticed that just doing this:

Code:

ping6 -v fe80::1%eth0 PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0 32 bytes from fe80::1%eth0: Neighbor Advertisement 32 bytes from fe80::1%eth0: Neighbor Advertisement ^C --- fe80::1%eth0 ping6 statistics --- 11 packets transmitted, 0 packets received, 100.0% packet loss

...usually causes the router to magically show up in ndp -aand everything starts working! (I mean, I've never gotten a ping response from it, even when things are working:

Code:

[shn ~] -> ping6 www.kame.net PING6(56=40+8+8 bytes) 2605:a140:2114:2385::23 --> 2001:2f0:0:8800:226:2dff:fe0b:4311 16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=0 hlim=47 time=164.888 ms 16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=1 hlim=47 time=164.951 ms 16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=2 hlim=47 time=164.620 ms ^C --- mango.itojun.org ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 164.620/164.820/164.951/0.143 ms [shn ~] -> ping6 fe80::1%eth0 PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0 ^C --- fe80::1%eth0 ping6 statistics --- 7 packets transmitted, 0 packets received, 100.0% packet loss

...but seeing the discovery packets come in with the -v switch to ping seems to indicate/change something. (Tells the router my host exists?)

So...is the problem on their end or mine? Is running ndp -a just reporting or does it change the system state? Does the ping6 -v "fix" give any indication what might be going on?

Requisite configs below. They don't change whether I have global connectivity or not.

Thanks all.

-- Ward

My rc.conf for this static configuration looks like this:

Code:

hostname=vmi1142385.contaboserver.net ifconfig_DEFAULT="DHCP inet6 accept_rtadv" growfs_enable="YES" cloudinit_enable="YES" # ------------------------------------------------------------------------------ ifconfig_vtnet0_name=eth0 ipv6_enabled="YES" firewall_enable="NO" defaultrouter=207.244.224.1 ifconfig_eth0="inet 207.244.238.92 netmask 255.255.240.0" ifconfig_eth0_alias0="inet 207.244.238.92 netmask 255.255.240.0" ifconfig_eth0_alias1="inet 144.126.129.136 netmask 255.255.240.0" ipv6_defaultrouter="fe80::1%eth0" ifconfig_eth0_ipv6="inet6 2605:a140:2114:2385::23 prefixlen 64" ifconfig_eth0_alias2="inet6 2605:a140:2114:2385::23 prefixlen 64" ifconfig_eth0_alias3="inet6 2605:a140:2114:2385::05 prefixlen 64"

and produces this interface:

Code:

[/etc] -> ifconfig eth0 eth0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> ether 00:50:56:49:64:a7 inet6 fe80::250:56ff:fe49:64a7%eth0 prefixlen 64 scopeid 0x1 inet6 2605:a140:2114:2385::23 prefixlen 64 inet6 2605:a140:2114:2385::5 prefixlen 64 inet 207.244.238.92 netmask 0xfffff000 broadcast 207.244.239.255 inet 144.126.129.136 netmask 0xfffff000 broadcast 144.126.143.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

And the routes:

Code:

Internet6: Destination Gateway Flags Netif Expire ::/96 ::1 UGRS lo0 default fe80::1%eth0 UGS eth0 ::1 link#2 UHS lo0 ::ffff:0.0.0.0/96 ::1 UGRS lo0 2605:a140:2114:2385::/64 link#1 U eth0 2605:a140:2114:2385::5 link#1 UHS lo0 2605:a140:2114:2385::23 link#1 UHS lo0 fe80::/10 ::1 UGRS lo0 fe80::%eth0/64 link#1 U eth0 fe80::250:56ff:fe49:64a7%eth0 link#1 UHS lo0 fe80::%lo0/64 link#2 U lo0 fe80::1%lo0 link#2 UHS lo0 ff02::/16 ::1 UGRS lo0

My server at Netcup had the same issue as well. The v6 route to our VPS would just suddenly not work. It's as though their stateful firewall keeps track of active v6 sessions to our VPS for XXs then kills the session, and when this happens I'd have to ping6 the VPS remotely which won't reply right away, but after a couple of ping attempts it suddenly replies then cycles through killing the session again after 30s - 1 min. Their 'fix' was to buy an additional /64 subnet which is ridiculous so I had a script that ping6 the defaultrouter fe80::1 every 20s until one day magically they fix it and things just started to work again after that.