External IP on lo0 (Why?)

surv

surv

Code: 
# uname -rs
FreeBSD 11.0-BETA2
After boot:
Code: 
# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS       wlan0
127.0.0.1          link#1             UH          lo0
192.168.1.0/24     link#2             U         wlan0
[COLOR=#ff0000]192.168.1.3        link#2             UHS         lo0[/COLOR]
If I add the line
010 allow log all from any to any via lo0
to ipfw config, in the logs I see it:
kernel: ipfw: 10 Accept TCP 201.51.27.28:43 192.168.1.3:60689 [COLOR=#ff0000]in[/COLOR] via [COLOR=#ff0000]lo0[/COLOR]
and similar connections from Internet IP's

This is normal ?
 
Yes. It's its own IP address. And there's only one destination to route it to; localhost.
 
It's normal on recent versions (same on 10.3) to see it in netstat -rn for the UHS routes. I believe the routing is setup like that to enable super-jumbo (yeah, I just made that term up) frames for local traffic which happens to use one of the external addresses, i.e. faster sockets / less overhead. lo0 has a 16384 MTU, so you get much higher performance. I'd need to dig deep into the code to be absolutely certain, but I believe it's just FreeBSD's way of essentially short-circuiting local traffic. I say "recent versions" as I'm not about to go back and dig through history; it probably has been like that for a long time.

I do not believe that it is generally good practice to attach IPFW rules involving external addresses to lo0 (they should be attached to the appropriate external interface). For best performance, lo0 should generally be minimised in firewall filtering or completely exempted from firewall processing (i.e. setup_loopback() from /etc/rc.firewall for IPFW, or set skip on lo0 for PF). There may be some quite specific circumstances where that is good and necessary (e.g. possibly some styles of jail / VM configs on a loopback network behind in-host NAT), just not something in a normal general/simple config.
 
SirDice said:
Yes. It's its own IP address. And there's only one destination to route it to; localhost.
Click to expand...
I thought that lo0 is intended only for "internal" use (me to me)

Murph said:
I believe the routing is setup like that to enable super-jumbo (yeah, I just made that term up) frames for local traffic which happens to use one of the external addresses, i.e. faster sockets / less overhead. lo0 has a 16384 MTU, so you get much higher performance.
Click to expand...
Perhaps this explains why almost all connections from Internet go to wlan0 and only some go to lo0
Murph said:
I do not believe that it is generally good practice to attach IPFW rules involving external addresses to lo0 (they should be attached to the appropriate external interface).
Click to expand...
Yes, but I added this rule just for logging of this connections
 
If, following handbook, you setup two jails on external ips an some external interface, jails will be able to talk to each other, because of set skip on lo0, even when everything else is blocked.

I have tried to come up with a set of rules that may allow jail1 to serve jail2 on one port with everything else blocked. I seem to either block everthing, or allow everything via set skip on lo0.

Are there reliable pf rules for such common scenario?
 
Murph said:
I do not believe that it is generally good practice to attach IPFW rules involving external addresses to lo0 (they should be attached to the appropriate external interface). For best performance, lo0 should generally be minimised in firewall filtering or completely exempted from firewall processing (i.e. setup_loopback() from /etc/rc.firewall for IPFW, or set skip on lo0 for PF). There may be some quite specific circumstances where that is good and necessary (e.g. possibly some styles of jail / VM configs on a loopback network behind in-host NAT), just not something in a normal general/simple config.
Click to expand...

pf rule:
Code: 
antispoof for lo0
 
Trihexagonal said:
pf rule:
Code: 
antispoof for lo0
Click to expand...
That is a host's IP on lo0, hence legitimate. May be antispoof can close it. Then connectivity is lost. Seems to be a good reason for telling everyone set skip on lo0.
Unfortunately this automagic host ip assign doesn't boil well with fortifying second perimeter in jails defense, i.e. when one of apps gets busted. Thread on related topic http://freebsd.1045724.x6.nabble.com/Firewalling-jails-and-lo0-td6120599.html Note how colorful is tcpdump.
 
SirDice said:
Yes. It's its own IP address. And there's only one destination to route it to; localhost.
Click to expand...
Can you explain why this always happens, or point to any useful explanation? This seems to be a glaring hole in understanding of how networking is handled.
Can we even have a special doc page in a handbook for lo0? It touches the second perimeter in jails' defense.
 
Murph said:
It's normal on recent versions (same on 10.3) to see it in netstat -rn for the UHS routes. I believe the routing is setup like that to enable super-jumbo (yeah, I just made that term up) frames for local traffic which happens to use one of the external addresses, i.e. faster sockets / less overhead. lo0 has a 16384 MTU, so you get much higher performance. I'd need to dig deep into the code to be absolutely certain, but I believe it's just FreeBSD's way of essentially short-circuiting local traffic. I say "recent versions" as I'm not about to go back and dig through history; it probably has been like that for a long time.

I do not believe that it is generally good practice to attach IPFW rules involving external addresses to lo0 (they should be attached to the appropriate external interface). For best performance, lo0 should generally be minimised in firewall filtering or completely exempted from firewall processing (i.e. setup_loopback() from /etc/rc.firewall for IPFW, or set skip on lo0 for PF). There may be some quite specific circumstances where that is good and necessary (e.g. possibly some styles of jail / VM configs on a loopback network behind in-host NAT), just not something in a normal general/simple config.
Click to expand...

I want to express that we shouldn't down-play this, by saying that it is "not something normal".
Anyone who had more than one jail on a host, has weakened network filtering defense.
 
Mikalai said:
I want to express that we shouldn't down-play this, by saying that it is "not something normal".
Anyone who had more than one jail on a host, has weakened network filtering defense.
Click to expand...
Jails provide process separation/isolation, not network separation.
 
SirDice said:
Jails provide process separation/isolation, not network separation.
Click to expand...
That what it smells like. At the time of jail creation, people were not planning on putting 20 jails that have parts from whole web stack, including horizontal splitting of mono-apps into micro services (it reduces blast radius on intrusion).
Localhost, I guess, is by some RFC treats all local traffic equally trusting.

I fill a little betrayed. There is zero warning signs about addresses being put to lo0. And there is tons of advertisement of jails' strong isolation. Except today, word isolation is also loaded with network separation, cause all processes are networked. In fact isolation, isolation with no leaks.
On another hand, it only has been in a last couple of years that we have more secure ipc flags and implementation for running things like Postgres. :(
 
You must log in or register to reply here.
Back
Top