Solved After Upgrading FreeBSD from 13.2 to 14.1: In-kernel NAT has issues with transporting network packages with big payload

[smithi]

Would it make sense to log the IPFW rules? After all I'm able to reach my Apache or other services in the jail via my NAT rules, so there isn't a rule that accidentally blocks something here.

It often makes sense if things are happening that you don't understand. Despite the erroneous handbook, syntax for unlimited logging to /var/log/security is eg
# ipfw add [$number] log [logamount 0] $body

Sometimes something useful is revealed.

 

[The_Educated_Fool]

Thanks for your responses!
I'll try over the weekend. tcp-setmss sounds promising.

About starting from scratch up: Unfortunately this happens on a rented virtual server. So it is not that easy to reproduce, since if I reconfigure the same on a spare machine at home, situation (hardware etc.) is very likely very different.
Renting a second VM would be a solution but I still hope to find the issue on my virtual server and save me the money...

Kind regards,
Fool

 

[The_Educated_Fool]

Hello,

seems like I solved my issue!
If you've followed this thread, you have noticed that I was able to narrow down the issue to either my IPFW rules or the in-kernel NAT I was using for my jails.
Since it is a long time ago that I've setup my firewall including the in-kernel NAT I started reading a bit about this again and found this thread here: https://forums.freebsd.org/threads/ipfw-nat-mystery.71351.
It said that TSO has to be switched off for in-kernel NAT. After executing

sysctl net.inet.tcp.tso=0

and also put that in
/etc/sysctl.conf, it worked again!

I'm still a little bit confused how my setup worked before my upgrade to FreeBSD 14.1; because I checked the snapshot of the old 13.2 boot environment and didn't find this setting their either. It doesn't seem to be removed by accident during my upgrade.
But the more important thing is that my server is again fully functional!

Thank you all for your help and hints! It is always a pleasure being here!

Kind regards,
Fool

PS: Does it make sense to somehow rename this threads topic for better helping other people in the future?
After all the problem is not Apache related...

 

[SirDice]

PS: Does it make sense to somehow rename this threads topic for better helping other people in the future?
After all the problem is not Apache related...

If you can. Editting old posts is limited due to some abuse we've had in the past. So you may not be able to. Let me know and I'll change it for you. Do you want me to move the thread to "Firewalls" too?

 

[smithi]

Since it is a long time ago that I've setup my firewall including the in-kernel NAT I started reading a bit about this again and found this thread here: https://forums.freebsd.org/threads/ipfw-nat-mystery.71351.
It said that TSO has to be switched off for in-kernel NAT. After executing

sysctl net.inet.tcp.tso=0

and also put that in
/etc/sysctl.conf, it worked again!

Oh wow ... just last week I had posted this regarding just that issue to someone else (and where it didn't help!)

Glad you hunted it down; ipfw(8) is very thorough, but also quite a lot to be famliar with.

 

[cracauer@]

I'm still a little bit confused how my setup worked before my upgrade to FreeBSD 14.1; because I checked the snapshot of the old 13.2 boot environment and didn't find this setting their either. It doesn't seem to be removed by accident during my upgrade.

Most likely your hard did not support TCP Segmentation Offload in 13.x.

 

[The_Educated_Fool]

@cracauer: Ah, that would explain things. Thanks!

SirDice: Seems like I was able to rename this thread. I hope the threads topic makes more sense now. If you have a better idea, feel free to change it.
And yes, please move it to "Firewalls".