IPv6 for the private LAN - but how?

PMc · Aug 18, 2021

The situation:

a lan with ~20 subnets connected together
a provider that offers a dynamic IPv4, a dynamic IPv6 /64, and optionally another dynamic IPv6 /56.

With IPv4 things are simple: all the subnets have private rfc1918 addresses, the uplink has a NAT, the NAT observes the address on the uplink interface, and each system can rightaway use outbound DNS, NTP, pkg repos or whatever. Whenever the dynamic address changes, NAT will immediately adapt to the new address and things continue to work.

With IPv6, as far as I understand there is no similar NAT. Instead, things are done with prefix delegation and DHCP. But then, when the router is restarted and the dynamic address (or prefix) changes, how will the client at the other end of the 20 subnets notice that it's former obtained address is no longer useful?

SirDice · Aug 19, 2021

PMc said:
Instead, things are done with prefix delegation and DHCP.

Or SLAAC.

PMc said:
But then, when the router is restarted and the dynamic address (or prefix) changes, how will the client at the other end of the 20 subnets notice that it's former obtained address is no longer useful?

That's what rtadvd(8) does on the router and rtsold(8) on the client. One 'advertises' router announcements and the other 'solicits' those advertisements.

PMc · Aug 19, 2021

SirDice said:
Or SLAAC.

I thought SLAAC is an autoconfiguration scheme?
The systems which I am mostly concerned with are rather of the server kind: they are well accounted for, and are locally accessed via (lan-local) DNS.

Specifically: If my desktop decides to obtain some address via SLAAC, how would the other nodes be able to find it (and open x-windows when they want to tell me something)?

SirDice said:
That's what rtadvd(8) does on the router and rtsold(8) on the client. One 'advertises' router announcements and the other 'solicits' those advertisements.

rtadvd sends router advertisement packets to the specified interfaces.
If no interfaces are specified, rtadvd will still run, but will not ad-
vertise any routes until interfaces are added using rtadvctl(8).

The program will daemonize itself on invocation. It will then send
router advertisement packets periodically, as well as in response to
router solicitation messages sent by end hosts.

Sorry, I don't get it. This sounds like it might take a while. This is the scheme I am concerned with:

Code:

          ----------------     ------------                      -------------
          | router1      |     | backbone |                      | cloud host|
          |              |     |          |                      |  offering |
(uplink) -|- WAN    LAN -|-----|-         |                      |   tunnel  |
          |dyn IP        |     |          |                      -------------
          ----------------     ------------                                .
                                |        |                                 .
            ----------------------     ----------------------              .
            |      bridge1       |     |     bridge2        |              .
            ----------------------     ----------------------              .
              |            |            |          |       |               .
           -------       -------      -------  ---------   ----------       .
           server1       server2      desktop   router2     (tunnel)  ...../
           -------       -------      -------  ---------   VPN-server
                                                   |       ----------
                                                  WLAN

When the uplink on router1 gets restarted and gets a different outbound IP/prefix,
- how do the WLAN clients behind router2 get the idea that they need a new address?
- how do the VPN clients get a new address?

And there is a more fancy part: I cannot run a VPN server on a dynamic IP. But I can tunnel-in a static IP from my cloud site, and then run the VPN on that static IP, to create the VPN-tunnels within that tunnel (mtu 1280). This is better than dynDNS, because the VPN sessions are not interrupted when the dynamic IP changes.
(I could switch to a failover uplink, while downloads through the tunnel would just stream along without noticing it.)

And I do not see how this would work with dynamic IPv6. (I rather get the idea that dynamic IPv6 is evil in itself.)

mumu · Aug 20, 2021

PMc said:
I thought SLAAC is an autoconfiguration scheme?
The systems which I am mostly concerned with are rather of the server kind: they are well accounted for, and are locally accessed via (lan-local) DNS.

Specifically: If my desktop decides to obtain some address via SLAAC, how would the other nodes be able to find it (and open x-windows when they want to tell me something)?

Sorry, I don't get it. This sounds like it might take a while. This is the scheme I am concerned with:

Code:

---------------- ------------ ------------- | router1 | | backbone | | cloud host| | | | | | offering | (uplink) -|- WAN LAN -|-----|- | | tunnel | |dyn IP | | | ------------- ---------------- ------------ . | | . ---------------------- ---------------------- . | bridge1 | | bridge2 | . ---------------------- ---------------------- . | | | | | . ------- ------- ------- --------- ---------- . server1 server2 desktop router2 (tunnel) ...../ ------- ------- ------- --------- VPN-server | ---------- WLAN

When the uplink on router1 gets restarted and gets a different outbound IP/prefix,
- how do the WLAN clients behind router2 get the idea that they need a new address?
- how do the VPN clients get a new address?

And there is a more fancy part: I cannot run a VPN server on a dynamic IP. But I can tunnel-in a static IP from my cloud site, and then run the VPN on that static IP, to create the VPN-tunnels within that tunnel (mtu 1280). This is better than dynDNS, because the VPN sessions are not interrupted when the dynamic IP changes.
(I could switch to a failover uplink, while downloads through the tunnel would just stream along without noticing it.)

And I do not see how this would work with dynamic IPv6. (I rather get the idea that dynamic IPv6 is evil in itself.)

Hi, ideally u want a static /56 from your provider not dynamic. Since u have a bunch of subnets, it will make IP numbering alot easier. You can assign each server it's own /64 while the clients get SLAAC addresses via the dynamic /64.

The one problem with dynamic /64 is when it changes, your client's will still use the old address rather than the new one. You should configure rtadvd or whichever SLAAC daemon u use to force the old addresses to be deprecated when the /64 changes.

PMc · Aug 30, 2021

mumu said:
Hi, ideally u want a static /56 from your provider not dynamic. Since u have a bunch of subnets, it will make IP numbering alot easier. You can assign each server it's own /64 while the clients get SLAAC addresses via the dynamic /64.

They have static, but they charge a solid premium and call it a "business solution".

What I get for standard is a dynamic /56. That is quite interesting - I can build subnets with that, but they need be assigned dynamically.

mumu said:
The one problem with dynamic /64 is when it changes, your client's will still use the old address rather than the new one. You should configure rtadvd or whichever SLAAC daemon u use to force the old addresses to be deprecated when the /64 changes.

I didn't find a switch to configure this, it seems to happen by default anyway.

But that's not yet the solution - rtadvd(8) will only talk to machines on the directly connected link.
What about the machines behind a 2nd or 3rd hop router? Somehow these 256 subnets must be distributed onwards, and then, when the /56 prefix changes, everybody behind the sternmost router must switch their address in an instant.

Alain De Vos · Aug 30, 2021

Interesting question.
I don't know if this is usable,

FreshPorts -- net/dhcp6: KAME DHCP6 client, server, and relay

WIDE-DHCPv6 is an open-source implementation of Dynamic Host Configuration Protocol for IPv6 (DHCPv6), originally developed by the KAME project. The implementation mainly conforms to the following standards: - RFC3315: Dynamic Host Configuration Protocol for IPv6 (DHCPv6) - RFC3319: DHCPv6...

www.freshports.org

PMc · Aug 30, 2021

Alain De Vos said:
Interesting question.
I don't know if this is usable,

FreshPorts -- net/dhcp6: KAME DHCP6 client, server, and relay

WIDE-DHCPv6 is an open-source implementation of Dynamic Host Configuration Protocol for IPv6 (DHCPv6), originally developed by the KAME project. The implementation mainly conforms to the following standards: - RFC3315: Dynamic Host Configuration Protocol for IPv6 (DHCPv6) - RFC3319: DHCPv6...

www.freshports.org

Ahh, yes, this is someway useable, but DHCP does not do what is requred - see discussion here.

That discussion is quite discouraging - some say that one needs to unplug(!) the network cable at the client side (of every client) to make it change the IP address. Others say that needs "only" power-cycle the (i.e. every) client. That's not even funny.

driesm · Aug 30, 2021

Its not that hard IMO, the easiest is if your ISP uses DHCPv6 to give you a prefix. Then with for example the net/dhcpcd port, you can choose on which interface to put which prefix. Lets say you get a /56 global prefix, with the correct configuration of dhcpcd, you can tell it to put a /64 sub prefix on bridge1, and another /64 sub prefix on bridge2. Then just make sure ratdvd is running on those interfaces. Any client connected on those bridges that uses rtsol will obtain an IPv6 address from the specified prefix, so basically any client that is IPv6 capable. If your prefix would change because of the dynamic nature, net/dhcpcd will update the prefixes on the interfaces and rtadvd will announce the new prefix all automagically. If you want I can share an example dhcpcd.conf for this configuration.

If you want to define access control / DNS host names its perfectly acceptable to put an extra private /64 IPv6 subnet on the LAN/bridge interfaces. This way any client will obtain a "global address" and a "private address". Global addresses can change, the private ones will not as you hard code that like you do for IPv4 today. Rtadvd automatically announces any prefix defined on the interface. These ULA addresses will be static based on the MAC address of the client (for eg DNS host names in your local LAN'(s)).

There is an analog of NAT for IPv6, typically called NPTv6 (only IPFW firewall). I have successfully deployed this approach in the past (only ULA addresses on the LAN) but there are some tricky parts with it. I have since reverted to the global + private subnet approach. The biggest reason for me was that Windows clients do not prefer ULA IPv6 addresses over IPv4 private addresses. Meaning that Windows will default to IPv4 even though IPv6 is deployed with NPTv6.

mumu · Aug 30, 2021

PMc said:
They have static, but they charge a solid premium and call it a "business solution".

What I get for standard is a dynamic /56. That is quite interesting - I can build subnets with that, but they need be assigned dynamically.

Yeh this is bad. As mentioned ideally u want a static /56. My ISP has the same problem. I'm trying to convince them to switch to assigning a static /56 instead.

PMc said:
I didn't find a switch to configure this, it seems to happen by default anyway.

But that's not yet the solution - rtadvd(8) will only talk to machines on the directly connected link.
What about the machines behind a 2nd or 3rd hop router? Somehow these 256 subnets must be distributed onwards, and then, when the /56 prefix changes, everybody behind the sternmost router must switch their address in an instant.

You can play with the 'pltime' or Preferred Lifetime in rtadvd.conf and reduce the value to for example, 10s. However you will need to test this in an isolated env. So with and without pltime, for the test a simple ping6 from an isolated client then reconnect your WAN to get a new dynamic /56 and see what happens on the test client. Does the ping6 resume and if not, you cancel the ping6 and run it again immediately, do you get replies ?

If somehow magically u get a static /56, just reset/remove pltime from rtadvd.conf so it reverts to the default value (7 days). Good luck.

mumu · Aug 30, 2021

PMc said:
That discussion is quite discouraging - some say that one needs to unplug(!) the network cable at the client side (of every client) to make it change the IP address. Others say that needs "only" power-cycle the (i.e. every) client. That's not even funny.

That's not something I'd do either lol. I ended up just disabling PD request on the router but still get an IPv6 on the WAN interface and just use ULA addresses on the LAN side and run PF nat just like for IPv4.

PMc · Aug 30, 2021

Duffyx said:
Its not that hard IMO, the easiest is if your ISP uses DHCPv6 to give you a prefix. Then with for example the net/dhcpcd port, you can choose on which interface to put which prefix. Lets say you get a /56 global prefix, with the correct configuration of dhcpcd, you can tell it to put a /64 sub prefix on bridge1, and another /64 sub prefix on bridge2. Then just make sure ratdvd is running on those interfaces. Any client connected on those bridges that uses rtsol will obtain an IPv6 address from the specified prefix, so basically any client that is IPv6 capable. If your prefix would change because of the dynamic nature, net/dhcpcd will update the prefixes on the interfaces and rtadvd will announce the new prefix all automagically. If you want I can share an example dhcpcd.conf for this configuration.

Very well, this is quite exactly what I am currently building.

But there is still a substantial missing link here:

Yes, I am getting the ISP prefix via DHCPv6
Yes, I am running dhcpcd.
But it has to run on router1 where the uplink is - because DHCPv6 uses link-local addresses.
Yes, dhcpcd can put a sub prefix onto the other interfaces - but that would only be the
LAN interface on router1. (see the picture above!)
Yes, rtadvd will then communicate the new address to clients on the same link. And that would mean only the (lefthand) inbound interface of the backbone, but NOT bridge1 and 2.

But I think I have figured something to make this work ... just need to write it down and check if it is consistent...

Duffyx said:
If you want to define access control / DNS host names its perfectly acceptable to put an extra private /64 IPv6 subnet on the bridge interfaces. This way any client will obtain a "global address" and a "private address". Global addresses can change, the private ones will not as you hard code that like you do for IPv4 today. Rtadvd automatically announces any prefix defined on the interface. These ULA addresses will be static based on the MAC address of the client (for eg DNS host names in your local LAN'(s)).

There is an analog of NAT for IPv6, typically called NPTv6 (only IPFW firewall). I have successfully deployed this approach in the past (only ULA addresses on the LAN) but there are some tricky parts with it. I have since reverted to the global + private subnet approach. The biggest reason for me was that Windows clients do not prefer ULA IPv6 addresses over IPv4 private addresses. Meaning that Windows will default to IPv4 even though IPv6 is deployed with NTPv6.

Yes, this is what I probably will do with the tunnels.

driesm · Aug 31, 2021

Woops, I initially misinterpreted the schema and though that bridge1/2 where virtual interfaces on the host so dhcpcd could touch them, NVM.
I agree with all your points. Just a question about the last.

PMc said:
Yes, rtadvd will then communicate the new address to clients on the same link. And that would mean only the (lefthand) inbound interface of the backbone, but NOT bridge1 and 2.
But I think I have figured something to make this work ... just need to write it down and check if it is consistent...

What exactly are the "backbone" and "bridge1/2"?
If they are just layer 2 switches than the rtadvd packets will just be broadcast through the switches to the clients behind them.
Its still the same link with respect to the LAN interface.

sko · Aug 31, 2021

You can't use a dynamically assigned prefix for your internal network without major breakage each time the ISP decides to assign you another prefix. period.

If they only offer you that crippled IPv6 solution you basically have only 2 options: dump that ISP (that ignores standards anyways by delegating a /56 instead of the required /48) or stay with IPv4 on the internal side (which is perfectly fine).
You might find some horrible how-tos about non-standard stuff like NATing internal IPv6 networks to public v6 addresses, but _NEVER_ do that. Just stay with simple, clean IPv4 on your internal network (lots of "smart" stuff and a bunch of windows software still doesn't support IPv6 or implements it wrong anyways) and let your gateway handle NATing to your external IPs (v4 and v6).
You could also go IPv6-only on the public side (and have fun with all the breaking due to the massive amount of services/servers still not accessible via v6) and stay IPv4 on the internal side - this is how probably most home and SMB-networks will look for the next ~20-30 years which is perfectly fine as IPv4 is and will NOT be deprecated in the foreseeable future.

IPv6 on the LAN today mostly makes sense if you have publicly available services running; for networks entirely populated with clients an local services, just stick with IPv4...

PMc · Aug 31, 2021

Duffyx said:
Woops, I initially misinterpreted the schema and though that bridge1/2 where virtual interfaces on the host so dhcpcd could touch them, NVM.
I agree with all your points. Just a question about the last.

What exactly are the "backbone" and "bridge1/2"?
If they are just layer 2 switches than the rtadvd packets will just be broadcast through the switches to the clients behind them.
Its still the same link with respect to the LAN interface.

router and backbone are FreeBSD instances, only the bridges are ng_bridge().

And I like Your idea. I thought for a moment, it might be possible to flatten the whole net, i.e. handle it mostly on layer 2. But I then didn't pursue that thought to the end. And You are right, that would have made this matter much easier. (But to actually change it would need to change the currently running IPv4 logic alongside, so it would be a serious effort.)

PMc · Aug 31, 2021

sko said:
You can't use a dynamically assigned prefix for your internal network without major breakage each time the ISP decides to assign you another prefix. period.

That is correct, and that is not intended. For the internal communcations there is the fd:: prefix.
But then, lots of machines want to talk to the outside also, for NTP, DNS, pkg security, VoIP, or simply websurfing. These are all short-lived connections, and they will be retried on failure, so I don't think there is a major breakage.
But in order to enable them to do that, they need a IPv6 GU address (or some 6to4- or 4to6-whatever, which I didn't bother to look into at all).

sko said:
If they only offer you that crippled IPv6 solution you basically have only 2 options: dump that ISP (that ignores standards anyways by delegating a /56 instead of the required /48) or stay with IPv4 on the internal side (which is perfectly fine).

It doesn't matter if the internal network runs IPv4 or IPv6. Some machine somewhere on the LAN will want to connect to some cloud host or whatever that has IPv6-only (and these are offered nowadays), and then it needs an address to do that. And for that to work the LAN has to be able to route IPv6 and provide routeable GU addresses all through the network. That's the whole point in this.

sko said:
You might find some horrible how-tos

Yes, you will get one of these "horrible" tutorials now, from me.
I didn't find a proper one, I only found people stating that what I am doing is "impossible" or such, so I had to do it myself. Enjoy, like, share and comment.

sko said:
You could also go IPv6-only on the public side (and have fun with all the breaking due to the massive amount of services/servers still not accessible via v6)

Aye, the IPMI of my cloud provider does only run IPv4. So this would be the end already.
But gladly, I am now fully operative with the entire LAN on dualstack.

sko said:
and stay IPv4 on the internal side - this is how probably most home and SMB-networks will look for the next ~20-30 years

That may be, unless they read my tutorial.

sko said:
IPv6 on the LAN today mostly makes sense if you have publicly available services running; for networks entirely populated with clients an local services, just stick with IPv4...

See above. For publicly accessible services you shoud indeed get a static GU IPv6. But the point you currently don't seem to recognise is that also for outgoing connections to an IPv6-only site we need a public address (which not necessarily need to be static) - or some tunneling/converter/whatever-creepy-stuff.

PMc · Aug 31, 2021

mumu said:
Yeh this is bad. As mentioned ideally u want a static /56. My ISP has the same problem. I'm trying to convince them to switch to assigning a static /56 instead.

Alright, hope You're lucky with that!

mumu said:
You can play with the 'pltime' or Preferred Lifetime in rtadvd.conf and reduce the value to for example, 10s. However you will need to test this in an isolated env.

I think I have now a solution that doesn't need twists. Fully standard and staying with all the defaults.
I measured it takes 16 sec. to reconnect, re-establish connectivity and distribute the IPs throughout the network, plus another 8 sec. each for a 3rd, 4th,... level of subordinate routers. One can probably tune that and make it faster, but I think one can live with that.

mumu · Sep 1, 2021

PMc said:
I think I have now a solution that doesn't need twists. Fully standard and staying with all the defaults.
I measured it takes 16 sec. to reconnect, re-establish connectivity and distribute the IPs throughout the network, plus another 8 sec. each for a 3rd, 4th,... level of subordinate routers. One can probably tune that and make it faster, but I think one can live with that.

You could also test with a client streaming YT/Netflix over IPv6 then re-connect WAN and see how quick it picks up the new address.

PMc · Sep 1, 2021

mumu said:
You could also test with a client streaming YT/Netflix over IPv6 then re-connect WAN and see how quick it picks up the new address.

YT plays all the time, it doesn't interrupt from that, only send more ads.

jem · Sep 2, 2021

My ISP provides native IPv6 /56 prefixes via DHCP and I've had it working on multiple home subnets, via my FreeBSD router, for a couple of years now. Here's an overview of my setup:

I have the KAME dhcp6 package installed, configured as follows (/usr/local/etc/dhcp6c.conf):

Code:

interface wan0 {
  send ia-pd 0;
};

id-assoc pd 0 {
  prefix ::/56 infinity;
  prefix-interface bridge0 {
    sla-id 0;
    sla-len 8;
  };
  prefix-interface lan4 {
    sla-id 1;
    sla-len 8;
  };
};

This requests a /56 prefix delegation from the ISP, then allocates /64 subnets from that /56 to my bridge0 and lan4 interfaces. The 'sla-id' parameter determines which subnet the interfaces get, for example, if the ISP had delegated 2001:db8:abcd:ef00::/56, then bridge0 would configure itself with the address 2001:db8:abcd:ef00:xxxx:xxff:fexx:xxxx/64 and lan4 would get 2001:db8:abcd:ef01:xxxx:xxff:fexx:xxxx, where the x's are the MAC address of the interface.

The router advertisement daemon (rtadvd) will advertise those /64 prefixes on those interfaces, declaring itself as the router. Any other IPv6 clients in the subnets configured to use SLAAC will then auto-configure themselves with IPv6 address in the relevant subnet and use the router as their IPv6 gateway.

The problem with this setup is the DHCP lease from my ISP only lasts an hour! If I'm careful to ensure the client renews the lease on time every time and never releases the lease, I always get the same prefix renewed and my LAN IPv6 addresses remain the same. If my router is offline for over an hour, it's likely I'll get a new prefix delegated and all my LAN IPs will change. Not useful.

To work around this, I also configure Unique Local Addresses (ULAs) on my LAN, alongside the global addresses my ISP delegated. I randomly chose a /48 prefix from the fd00::/8 range, then configure those on my subnets with the following rc.conf directives:

Code:

ipv6_prefix_bridge0="fdxx:xxxx:xxxx:0::"
ipv6_prefix_lan4="fdxx:xxxx:xxxx:1::"

rtadvd also advertises these prefixes to my LAN subnets, so clients configure themselves with one of those addresses as well. As this prefix is fixed, the LAN addresses will never change and I can manually set up DNS AAAA records for each host.

PMc · Sep 2, 2021

jem said:
My ISP provides native IPv6 /56 prefixes via DHCP and I've had it working on multiple home subnets, via my FreeBSD router, for a couple of years now. Here's an overview of my setup:

Ah, and these are dynamic, they change when you restart the computer/router? So this practice is not only a German invention?
And your subnets are all linklocal to the router? None that are reached via another hop?

jem said:
I have the KAME dhcp6 package installed, configured as follows (/usr/local/etc/dhcp6c.conf):

Yes, I tested that one: it basically works, but it seems unmaintained for some 8 or 9 years now, and it has some bugs.

jem said:
The problem with this setup is the DHCP lease from my ISP only lasts an hour! If I'm careful to ensure the client renews the lease on time every time and never releases the lease, I always get the same prefix renewed and my LAN IPv6 addresses remain the same.

That is cute. So you can reboot, install patches, etc. - only it must not last too long?!
I cannot do that - when I disconnect/reconnect, it's a different context and does know nothing of the previous lease.
Nevertheless I would not rely on such a construct (in the sense of putting such addresses into public name-servers or similar) - there could be an outage of whatever, while I'm not at home, and then I can't reach my own machines anymore.

mumu · Sep 3, 2021

jem said:
My ISP provides native IPv6 /56 prefixes via DHCP and I've had it working on multiple home subnets, via my FreeBSD router, for a couple of years now. Here's an overview of my setup:

Dynamic PD is only useful for maybe 1-2 host in the network. If u have more machines, renumbering becomes quite tedious.

jem said:
I have the KAME dhcp6 package installed, configured as follows (/usr/local/etc/dhcp6c.conf):

Code:

interface wan0 { send ia-pd 0; }; id-assoc pd 0 { prefix ::/56 infinity; prefix-interface bridge0 { sla-id 0; sla-len 8; }; prefix-interface lan4 { sla-id 1; sla-len 8; }; };

Here you could just request ia-pd on the WAN and skip bridge0/lan4 and use ULA for bridge0/lan4 instead.

jem said:
Code:

ipv6_prefix_bridge0="fdxx:xxxx:xxxx:0::" ipv6_prefix_lan4="fdxx:xxxx:xxxx:1::"

rtadvd also advertises these prefixes to my LAN subnets, so clients configure themselves with one of those addresses as well. As this prefix is fixed, the LAN addresses will never change and I can manually set up DNS AAAA records for each host.

Lastly add a NAT rule in pf for both /48 ULA:

Code:

nat on $wan_if inet6 from ! ($wan_if) to any -> ($wan_if)

PMc · Sep 3, 2021

mumu said:
Dynamic PD is only useful for maybe 1-2 host in the network. If u have more machines, renumbering becomes quite tedious.

As I have shown here it is no problem. I am doing it for ~15 nodes currently, but one can do it for a thousand nodes in the same way. It takes 15-25 seconds, and one does usually not notice it.

mumu · Sep 3, 2021

PMc said:
As I have shown here it is no problem. I am doing it for ~15 nodes currently, but one can do it for a thousand nodes in the same way. It takes 15-25 seconds, and one does usually not notice it.

Yeh I have read it. Comprehensive guide and thanks for writing it. For me I just prefer having static addresses assigned by ISP so I don't have to deal with the dynamic mess it'll inherit later on.

IPv6 for the private LAN - but how?

Administrator