Promiscuous interface problem

This morning I put 7.1 on a Via box with RealTek 8139C+/8169/8169S/8110S
Ethernet card (from sysinstall). I'm new to FreeBSD, but I've been
playing with this box to learn a little about the BSD side.

The network seems not to work unless the interface (re0) is in
promiscuous mode.

I don't understand this. arp -a looks reasonable, and it doesn't change
when promisc is turned off. ping works to localhost and to re0, but if I
start a ping to outside the machine, nothing happens until I enable
promisc by starting tcpdump or by issuing a command to ifconfig. Then
the echoes start. And disabling stops them again.

I don't think it's hardware because this box ran Debian Linux for
several months with no problems -- unless maybe the Debian driver kept
it in promisc all the time; I didn't look...

rc.conf:

Code:
# -- sysinstall generated deltas -- # Sun Mar 29 09:52:35 2009
# Created: Sun Mar 29 09:52:35 2009
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
hostname="otto.slsware.lan"
ifconfig_re0="inet 192.168.3.177  netmask 255.255.255.0"
defaultrouter="192.168.3.1"
font8x14="NO"
font8x16="swiss-8x16"
font8x8="swiss-8x8"
### linux_enable="YES"
### nfs_client_enable="YES"
### saver="daemon"
webmin_enable="YES"
# -- sysinstall generated deltas -- # Sun Mar 29 12:45:49 2009
gateway_enable="NO"
router_enable="YES"
ftpd_enable="YES"
inetd_enable="YES"
named_enable="YES"
ntpd_enable="YES"
postfix_enable="YES"
routed_enable="YES"
rsyncd_enable="YES"
sshd_enable="YES"

ifconfig -a:

Code:
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:30:18:a0:29:dc
	inet 192.168.3.177 netmask 0xffffff00 broadcast 192.168.3.255
	media: Ethernet autoselect (100baseTX <full-duplex>)
	status: active
re1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:30:18:a0:29:dd
	media: Ethernet autoselect (10baseT/UTP <half-duplex>)
	status: no carrier
plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
	inet6 ::1 prefixlen 128 
	inet 127.0.0.1 netmask 0xff000000


This is with promisc on. The output is identical with it off, except the
PROMISC flag is off.
 
ghe said:
The network seems not to work unless the interface (re0) is in
promiscuous mode.

I don't understand this. arp -a looks reasonable, and it doesn't change
when promisc is turned off. ping works to localhost and to re0, but if I
start a ping to outside the machine, nothing happens until I enable
promisc by starting tcpdump or by issuing a command to ifconfig. Then
the echoes start. And disabling stops them again.

Sort of sounds like a network misconfiguration. Run tcpdump(8) to capture icmp traffic next time you test this. Are the echo replies really coming back to your IP?
 
Yes, it certainly does sound like a misconfig, but I can't figure out what it is -- I don't know any of the tricks and gotchas of BSD network configuration, yet.

I did run tcpdump; that was my first clue. I'd started a ping, and nothing was happening. I went to another VTY, logged in, and ran tcpdump. It showed echo-request ICMP packets leaving my machine headed for the target, and echo-response packets coming back -- and when I went back to the original screen, pinging was happening. When I stopped tcpdump, the pings stopped.

The IP address was that of re0. And when the net is up, I can connect to the host (using the expected IP) with no problems.

The first pass at a post here said tcpdump had to be running to get the net to work, but when I added the ifconfig output to the post, I noticed the PROMISC flag. I tried again without tcpdump, but having ifconfig set PROMISC. It worked.

FreeBSD, installed as a virtual machine on the Mac (Sun VirtualBox) and seeing an 'em' flavor NIC, does *not* need the PROMISC flag. Of course, there are many other variables...

I ran nmap -sP on my LAN from another machine while the net wasn't working. nmap said my host was up. At least I think I did; I've tried a lot of stuff.

Another symptom: right after the minimal install, the net worked. I installed sssd and a couple other daemons, and the net quit. I commented out the enables in rc.conf, and the net started again.
 
So is the problem with the re(4) interface on a virtual machine or a real machine? If the latter, could you try a different NIC? One possibility is the driver isn't playing nicely with your particular RealTek NIC, and another (very small) possibility is that your NIC is beginning to fail. Might as well begin the process of elimination.

The behavior you're describing sounds very bizarre, but at least it is reproducible (if I am understanding you). So, perhaps submit a problem report at some point when you are satisfied that it's not a case of PEBKAC or failing hardware.
 
  • Thanks
Reactions: ghe
All the real machine. I'll try a different NIC -- there are 2 in the box; both act the same way.

It's reproducible, but I really don't feel like I know what I'm doing, so I'm almost trying things at random.

And as for bizarre... It looks very much like the problem may be that with promisc mode off, it isn't responding to who-has'es. With promisc on, I ssh into lanserver.slsware.lan (lsl) a couple times and start tcpdump -n in one session.

I turn promisc off on otto.slsware.lan (osl), and go back over to lsl. From there I ping osl, and nothing happens. The tcpdump shows lsl asking who-has for osl's IP and getting no response. It also shows lsl sending ssh packets to osl's IP. And, of course, if I break the ssh to lsl, I can't connect again (I think I've noticed FreeBSD stashing MACs in its routing table; Linux doesn't do that).

So
 
Swapping out the NIC did it. I'm reluctant to whine at the maintainers because I know so little about FreeBSD. But I think I'll drop them a note...

Thanks much.
 
ghe said:
Swapping out the NIC did it.

Great news.

ghe said:
I'm reluctant to whine at the maintainers because I know so little about FreeBSD. But I think I'll drop them a note...

You sound like you know more than enough to compose a comprehensible PR. ;)
 
Back
Top