Enabling IPv6 [eventually] freezes the gateway

dvl@

dvl@

Developer
Some time ago, I tried IPv6. I disabled it after it froze my gateway. I tried again today; it lasted 90 minutes.

Running FreeBSD 9.3-RELEASE #0

These are the lines I enabled in /etc/rc.conf

Code: 
ifconfig_em0_ipv6="inet6 2001:470:1f07:09ea:1::1"
ipv6_defaultrouter="2001:470:1f06:9ea::1"

All up, these are the entries which relate to IPv6

Code: 
$ grep -i ipv6 /etc/rc.conf
# our IPv6 tunnel
ipv6_activate_all_interfaces="YES"
ifconfig_gif0_ipv6="inet6 1f06:b80::2 2001:470:1f06:b80::1 prefixlen 128"
ifconfig_em0_ipv6="inet6 2001:470:1f07:09ea:1::1"
ipv6_defaultrouter="2001:470:1f06:9ea::1"
ipv6_gateway_enable="YES"
rtadvd_enable="YES"                     # let our LAN know the IPv6 default route

After making those changes, I rebooted the gateway. Immediately, I started seeing these messages:

Code: 
Sep  8 18:38:22 bast kernel: pf: state key linking mismatch! dir=OUT, if=em0, stored af=2, a0: 209.51.161.14, a1: 108.52.41.15, proto=41, found af=28, a0: 2001:470:1f07:9ea:4893:e490:edd1:1b8d[55496], a1: 2607:f8b0:4004:809::1015[443], proto=6.

In that 90 minutes, approximately 46,000 of these messages appeared. That's roughly 7 per second.

TIme to turn off IPv6 again...
 
FYI

Code: 
[root@bast:/usr/home/dan] # pfctl -s info
No ALTQ support in kernel
ALTQ related functions disabled
Status: Enabled for 0 days 00:39:12           Debug: Urgent

Interface Stats for fxp0              IPv4             IPv6
  Bytes In                        36985445                0
  Bytes Out                       12326189                0
  Packets In
    Passed                           66169                0
    Blocked                            314                0
  Packets Out
    Passed                           62727                0
    Blocked                              0                0

State Table                          Total             Rate
  current entries                      826               
  searches                          304508          129.5/s
  inserts                            12570            5.3/s
  removals                           11744            5.0/s
Counters
  match                              14096            6.0/s
  bad-offset                             0            0.0/s
  fragment                              87            0.0/s
  short                                  0            0.0/s
  normalize                              0            0.0/s
  memory                                 0            0.0/s
  bad-timestamp                          0            0.0/s
  congestion                             0            0.0/s
  ip-option                              0            0.0/s
  proto-cksum                            0            0.0/s
  state-mismatch                         0            0.0/s
  state-insert                           0            0.0/s
  state-limit                            0            0.0/s
  src-limit                              0            0.0/s
  synproxy                               0            0.0/s
 
I've found that some advanced PF stuff can interfere with HE.net tunnels. In short, if you use scrub rules, don't use reassemble tcp and if I remember right I think modulate state caused issues for me as well. Also don't forget a rule to allow in the inbound proto 41 traffic. Can you just show your /etc/pf.conf or an pfctl -s rules. Also, mention what interface is the WAN. Is em0 the LAN?
 
I did find:

I do not see reassemble in my rules, but I did find a modulate.

Code: 
pass out quick on $ext_if proto tcp all modulate state flags S/SA

Hmm.

I don't know about proto 41; I'm not seeing it in my ruleset.

The WAN is fxp0. The LAN is em0.

I have removed the following line from /etc/rc.conf:

Code: 
ipv6_activate_all_interfaces="YES"
 
Do not use modulate state unless you really have to. The only time when you have to use is when you are under a SYN flood attack as an active countermeasure. At other times it's best turned off.
 
Noted. At this point, I'm going to turn off one item at a time, to determine the cause. Thanks.
 
kpa said:
Do not use modulate state unless you really have to. The only time when you have to use is when you are under a SYN flood attack as an active countermeasure. At other times it's best turned off.
Click to expand...

I believe synproxy state is what you are thinking of here.
 
junovitch said:
kpa said:
Do not use modulate state unless you really have to. The only time when you have to use is when you are under a SYN flood attack as an active countermeasure. At other times it's best turned off.
Click to expand...

I believe synproxy state is what you are thinking of here.
Click to expand...

Ah yes, you are correct :r

Still, the out of the ordinary options are the first ones to check in troubleshooting.
 
Here is what I have that should be directly relevant to getting a gif interface up on FreeBSD 10.0. Do note that gif interface have to be configured a bit differently on FreeBSD 10.0.

/etc/rc.conf
Code: 
ifconfig_em3="SYNCDHCP"
cloned_interfaces="gif0"
ifconfig_gif0="tunnel x.x.x.83 216.66.22.2 mtu 1480"
ifconfig_gif0_ipv6="inet6 2001:x:x:x::2 2001:x:x:x::1 prefixlen 128"
ipv6_defaultrouter="2001:x:x:x::1"

/etc/pf.conf
Code: 
scrub on em3
scrub on gif0
nat on em3 inet from !(em3) to any -> (em3:0)
block log all
pass out keep state
pass in on $lan_ifs
pass in on em3 inet proto 41 from 216.66.22.2 to (em3:0)

dvl@ said:
I don't know about proto 41; I'm not seeing it in my rulesest.
Click to expand...
The IPv6 inside an IPv4 is protocol 41. Protocol 6 and 17 are the most common as TCP and UDP.
 
You don't even have to remember that IPv6 encapsulated in IPv4 is protocol number 41. You can use ipv6 as the protocol name directly because it's resolved from /etc/protocols:

Code: 
pass in log quick on $ext_if inet proto ipv6 from $pop_ip to ($ext_if)
 
dvl@ said:
I have removed the following line from /etc/rc.conf:

Code: 
ipv6_activate_all_interfaces="YES"
Click to expand...

With that single change, the gateway stayed running all night.
 
kpa said:
You don't even have to remember that IPv6 encapsulated in IPv4 is protocol number 41. You can use ipv6 as the protocol name directly because it's resolved from /etc/protocols:

Code: 
pass in log quick on $ext_if inet proto ipv6 from $pop_ip to ($ext_if)
Click to expand...

Ahh, I have:

Code: 
pass quick proto ipv6 all keep state

Covered... although perhaps I could lock that now just a tad bit more...
 
I forgot to mention: it appears that /var/log/messages continues to accumulate the previously mentioned entries even after the gateway becomes unresponsive via both keyboard and network.
 
kpa said:
junovitch said:
kpa said:
Do not use modulate state unless you really have to. The only time when you have to use is when you are under a SYN flood attack as an active countermeasure. At other times it's best turned off.
Click to expand...

I believe synproxy state is what you are thinking of here.
Click to expand...

Ah yes, you are correct :r

Still, the out of the ordinary options are the first ones to check in troubleshooting.
Click to expand...

FYI: My current rules use enither synproxy nor modulate.
 
dvl@ said:
I did find:

I do not see reassemble in my rules, but I did find a modulate.

Code: 
pass out quick on $ext_if proto tcp all modulate state flags S/SA
Click to expand...

I just dumped my rules. I found a reassemble, but it's not in the rule set. Only the first part of the rule set is shown here. Look for scrub in all fragment reassemble below:

Code: 
$ sudo pfctl -s rules
Password:
No ALTQ support in kernel
ALTQ related functions disabled
scrub in all fragment reassemble
block drop all
pass out quick on tun0 proto tcp all flags S/SA keep state
pass out quick on tun0 proto udp all keep state
...

Confirming that is not in my rule set:

$ grep -i assemb /etc/pf.conf
$
 
junovitch said:
Here is what I have that should be directly relevant to getting a gif interface up on FreeBSD 10.0. Do note that gif interface have to be configured a bit differently on FreeBSD 10.0.

/etc/rc.conf
Code: 
ifconfig_em3="SYNCDHCP"
cloned_interfaces="gif0"
ifconfig_gif0="tunnel x.x.x.83 216.66.22.2 mtu 1480"
ifconfig_gif0_ipv6="inet6 2001:x:x:x::2 2001:x:x:x::1 prefixlen 128"
ipv6_defaultrouter="2001:x:x:x::1"
Click to expand...

Here is what I have:

Code: 
ifconfig_fxp0="SYNCDHCP"
dan@bast:~] $ grep gif0 /etc/rc.conf
gif_interfaces="gif0"
gifconfig_gif0="96.x.x.96 209.51.161.14"
ifconfig_gif0_ipv6="inet6 1f06:x::2 2001:x:x:x::1 prefixlen 128"
ifconfig_em0_ipv6="inet6 2001:x:x:x:1::1"
ipv6_defaultrouter="2001:x:x:x::1"
ipv6_gateway_enable="YES"
rtadvd_enable="YES"
 
Are you sure your addresses are all right? I would double check all them. It looks like you are missing the prefixing 2001:470 here:
Code: 
ifconfig_gif0_ipv6="inet6 1f06:b80::2 2001:470:1f06:b80::1 prefixlen 128"

This isn't even in the same /64 subnet as your gif tunnel. It should match the second IPv6 address above.
Code: 
ipv6_defaultrouter="2001:470:1f06:9ea::1"
 
Also make sure that the IPv6 address you assign to the LAN interface is from a different prefix than the addresses on the gif0 interface. Mixing those up will prevent any kind of routing happening.
 
junovitch said:
Are you sure your addresses are all right? I would double check all them. It looks like you are missing the prefixing 2001:470 here:
Code: 
ifconfig_gif0_ipv6="inet6 1f06:b80::2 2001:470:1f06:b80::1 prefixlen 128"

This isn't even in the same /64 subnet as your gif tunnel. It should match the second IPv6 address above.
Code: 
ipv6_defaultrouter="2001:470:1f06:9ea::1"
Click to expand...

Thank you for raising that. My brain is fried and I need food. I've just made that change via the command line. Let's see how it runs.
 
kpa said:
Also make sure that the IPv6 address you assign to the LAN interface is from a different prefix than the addresses on the gif0 interface. Mixing those up will prevent any kind of routing happening.
Click to expand...

The internal net NIC (em0) has 2001:470:1f07:x:x::1 and gif0 has 2001:470:1f06:x::2 --> 2001:470:1f06:x::1

My thanks in helping me find this error.

Any speculation as to why that might be freezing stuff?
 
junovitch said:
Here is what I have that should be directly relevant to getting a gif interface up on FreeBSD 10.0. Do note that gif interface have to be configured a bit differently on FreeBSD 10.0.

/etc/rc.conf
Code: 
ifconfig_em3="SYNCDHCP"
cloned_interfaces="gif0"
ifconfig_gif0="tunnel x.x.x.83 216.66.22.2 mtu 1480"
ifconfig_gif0_ipv6="inet6 2001:x:x:x::2 2001:x:x:x::1 prefixlen 128"
ipv6_defaultrouter="2001:x:x:x::1"
Click to expand...

After several reboot in the past 3 hours, I've started using a setup similar to the above. What is the point of specifying mtu here?

My current setup is:

Code: 
# egrep -i 'ipv6|gif' /etc/rc.conf
cloned_interfaces="tun0 gif0"
ifconfig_gif0="tunnel 96.245.x.x 209.51.161.14 mtu 1480"
ifconfig_gif0_ipv6="inet6 2001:470:1f06:x::2 2001:470:1f06:9ea::1 prefixlen 128"
ifconfig_em0_ipv6="inet6 2001:470:1f07:x:1::1"
ipv6_defaultrouter="2001:470:1f06:9ea::1"
ipv6_gateway_enable="YES"
rtadvd_enable="YES"
 
Hi dvl@, Did You actually solve the problem? I'm facing this freeze too; in my case, without even going down the gif path, just by doing ULAs (fd00::... ) in the internal interface (which happens to be a bridge, bridging LAN and WLAN).
The freeze happens quite immediately when trying to send almost any kind of packet (ping6 or trying to telnet to some port) from one fd00::xxx address of a client, to the router.
The machine in question is currently running 10.2-STABLE #1 r290167, on i386.
Out of curiosity: is/was yours also running on i386, or amd64?
Thanx for any hint..
 
You must log in or register to reply here.
Back
Top