After upgrading all ports to 2023Q2, my site can no longer connect to www.freebsd.org, no matter which client used.
Analyzing shows that the problem is on the router. This is what arrives from the Internet:
And this is what gets routed onwards into the LAN:
As one can see, the three full-size packets (seq 1:3931) are not routed onwards, they apparently disappear without a hint. The client appropriately replies with SACK, and things are stuck.
Now I'm wondering which of the upgraded pkgs would change the routing layer in the kernel to start filtering full-size packets?
The router itself was upgraded to 13.2-BETA2 in February, and did work until yesterday - I was online in the forum yesterday. So the MTU was correct until then.
My cloud stuff is also offline, because suricata keeps crashing, with this error message (never seen before):
pkg[46971]: suricata upgraded: 6.0.9_1 -> 6.0.9_4
And this should not be a functional change (only changes to the rust compiler). So who is the culprit?
Analyzing shows that the problem is on the router. This is what arrives from the Internet:
Code:
17:36:19.702589 IP6 *****.30638 > 2610:1c1:1:606c::50:25.443: Flags [S], seq 3191615110, win 65535, options [mss 1322,nop,wscale 6,sackOK,TS val 2004750914 ecr 0], length 0
17:36:19.939529 IP6 2610:1c1:1:606c::50:25.443 > *****.30638: Flags [S.], seq 621848947, ack 3191615111, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4193283050 ecr 2004750914], length 0
17:36:20.143195 IP6 *****.30638 > 2610:1c1:1:606c::50:25.443: Flags [.], ack 1, win 1033, options [nop,nop,TS val 2004751354 ecr 4193283050], length 0
17:36:20.179993 IP6 *****.30638 > 2610:1c1:1:606c::50:25.443: Flags [P.], seq 1:518, ack 1, win 1033, options [nop,nop,TS val 2004751389 ecr 4193283050], length 517
17:36:20.417337 IP6 2610:1c1:1:606c::50:25.443 > *****.30638: Flags [.], seq 1:1311, ack 518, win 1033, options [nop,nop,TS val 4193283527 ecr 2004751389], length 1310
17:36:20.417429 IP6 2610:1c1:1:606c::50:25.443 > *****.30638: Flags [.], seq 1311:2621, ack 518, win 1033, options [nop,nop,TS val 4193283527 ecr 2004751389], length 1310
17:36:20.417458 IP6 2610:1c1:1:606c::50:25.443 > *****.30638: Flags [.], seq 2621:3931, ack 518, win 1033, options [nop,nop,TS val 4193283527 ecr 2004751389], length 1310
17:36:20.417505 IP6 2610:1c1:1:606c::50:25.443 > *****.30638: Flags [P.], seq 3931:4097, ack 518, win 1033, options [nop,nop,TS val 4193283527 ecr 2004751389], length 166
17:36:20.417519 IP6 2610:1c1:1:606c::50:25.443 > *****.30638: Flags [P.], seq 4097:4259, ack 518, win 1033, options [nop,nop,TS val 4193283527 ecr 2004751389], length 162
17:36:20.626034 IP6 *****.30638 > 2610:1c1:1:606c::50:25.443: Flags [.], ack 1, win 1033, options [nop,nop,TS val 2004751834 ecr 4193283050,nop,nop,sack 1 {3931:4097}], length 0
17:36:20.626960 IP6 *****.30638 > 2610:1c1:1:606c::50:25.443: Flags [.], ack 1, win 1033, options [nop,nop,TS val 2004751839 ecr 4193283050,nop,nop,sack 1 {3931:4259}], length 0And this is what gets routed onwards into the LAN:
Code:
17:35:55.331246 IP6 *****.54008 > 2610:1c1:1:606c::50:25.443: Flags [S], seq 2436214429, win 65535, options [mss 1322,nop,wscale 6,sackOK,TS val 2120056263 ecr 0], length 0
17:35:55.567001 IP6 2610:1c1:1:606c::50:25.443 > *****.54008: Flags [S.], seq 1903929895, ack 2436214430, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 3312350777 ecr 2120056263], length 0
17:35:55.771004 IP6 *****.54008 > 2610:1c1:1:606c::50:25.443: Flags [.], ack 1, win 1033, options [nop,nop,TS val 2120056703 ecr 3312350777], length 0
17:35:55.813145 IP6 *****.54008 > 2610:1c1:1:606c::50:25.443: Flags [P.], seq 1:518, ack 1, win 1033, options [nop,nop,TS val 2120056738 ecr 3312350777], length 517
17:35:56.049592 IP6 2610:1c1:1:606c::50:25.443 > *****.54008: Flags [P.], seq 3931:4097, ack 518, win 1033, options [nop,nop,TS val 3312351259 ecr 2120056738], length 166
17:35:56.049611 IP6 2610:1c1:1:606c::50:25.443 > *****.54008: Flags [P.], seq 4097:4258, ack 518, win 1033, options [nop,nop,TS val 3312351259 ecr 2120056738], length 161
17:35:56.258287 IP6 *****.54008 > 2610:1c1:1:606c::50:25.443: Flags [.], ack 1, win 1033, options [nop,nop,TS val 2120057188 ecr 3312350777,nop,nop,sack 1 {3931:4097}], length 0
17:35:56.262829 IP6 *****.54008 > 2610:1c1:1:606c::50:25.443: Flags [.], ack 1, win 1033, options [nop,nop,TS val 2120057188 ecr 3312350777,nop,nop,sack 1 {3931:4258}], length 0As one can see, the three full-size packets (seq 1:3931) are not routed onwards, they apparently disappear without a hint. The client appropriately replies with SACK, and things are stuck.
Now I'm wondering which of the upgraded pkgs would change the routing layer in the kernel to start filtering full-size packets?
Code:
pkg[89327]: pkg upgraded: 1.19.0 -> 1.19.1_1
pkg[89373]: libnghttp2 upgraded: 1.48.0 -> 1.52.0
pkg[89373]: protobuf upgraded: 3.21.9,1 -> 3.21.12,1
pkg[89373]: ca_root_nss upgraded: 3.86 -> 3.89
pkg[89373]: bind-tools upgraded: 9.18.12 -> 9.18.13
pkg[89373]: curl upgraded: 7.87.0 -> 7.88.1
pkg[89373]: lsof upgraded: 4.96.5,8 -> 4.97.0,8
pkg[89373]: openvpn upgraded: 2.5.9 -> 2.6.2
pkg[89373]: ruby upgraded: 3.0.5,1 -> 3.0.5_1,1
pkg[89373]: mutt upgraded: 2.2.9 -> 2.2.10
pkg[89373]: emacs-nox upgraded: 28.2_2,3 -> 28.2_4,3
pkg[89373]: bind916 upgraded: 9.16.38 -> 9.16.39
pkg[89373]: git upgraded: 2.39.2 -> 2.40.0The router itself was upgraded to 13.2-BETA2 in February, and did work until yesterday - I was online in the forum yesterday. So the MTU was correct until then.
My cloud stuff is also offline, because suricata keeps crashing, with this error message (never seen before):
Code:
suricata[39932]: [110405] <Warning> -- [ERRCODE: SC_WARN_IPFW_XMIT(84)] - Write to ipfw divert socket failed: Message too long
suricata[39932]: [109685] <Error> -- [ERRCODE: SC_ERR_FATAL(171)] - thread W-8677 failedpkg[46971]: suricata upgraded: 6.0.9_1 -> 6.0.9_4
And this should not be a functional change (only changes to the rust compiler). So who is the culprit?
