Problems with "No buffer space available"

O

oliver@

Developer
Hi,

after some uptime, my systems just starts out of nowhere logging:

Code: 
Feb 11 19:18:20 <local4.err> reis dhcpd: send_packet: No buffer space available
Feb 11 19:18:20 <local4.err> reis dhcpd: dhcp.c:3763: Failed to send 306 byte long packet over igb0 interface.
Feb 11 19:18:23 <local4.err> reis dhcpd: send_packet: No buffer space available
Feb 11 19:18:23 <local4.err> reis dhcpd: dhcp.c:3763: Failed to send 306 byte long packet over igb0 interface.
Feb 11 19:18:46 <local4.err> reis dhcpd: send_packet: No buffer space available
Feb 11 19:18:46 <local4.err> reis dhcpd: dhcp.c:3763: Failed to send 306 byte long packet over igb0 interface.

from this point on it continues to flood my log. After some days, the system just reboots out of nothing. Then it works again for around 3 days and then it starts again....

What the hell is going one?

Code: 
root@reis /> uptime
 7:36PM  up 3 days, 48 mins, 1 user, load averages: 0.15, 0.21, 0.21
root@reis /> uname -a
FreeBSD host 10.3-PRERELEASE FreeBSD 10.3-PRERELEASE #0: Sat Jan 16 14:51:18 CET 2016     user@host:/usr/obj/usr/src/sys/GENERIC  amd64
root@reis />
 
Firewall is fine and DHCP in general is working.
Firewall would also not explain the 3 days period without any messages. For me it looks like something is accumulating the 3 days until it is "full" and then starts bailing out this error messages
 
I wonder if the BMC is interfering on igb0. On my AST2400 board igb0 is shared with BMC.

Have you recently upgraded the Aspeed firmware?
 
Nope - I didn't do any board related software updates (BIOS, Firmware and so on) for at least 1.5 years
 
Have you changed the BMC password?
These boards are a leaking sieve with default BMC passwords like admin/password.

Are you internet facing with this machine?
 
The system just sits there in the basement for over a year barely touched (just FreeBSD + ports updates).
This system is connected to my VDSL router and routs my network. I'm seeing this behaviour for arround 1 month now.
I'll go and update FreeBSD now to the latest 10-STABLE. If the problem still exists I'll do ICMP and BIOS updates hoping that it will fix it.

BMC is untouched by me.
 
Are you aware Intel released an errata saying thier C2xxx chips could die after 18months?
https://forums.freebsd.org/threads/59663/

The problem with an untouched BMC is the thing has a web interface that is always available, even if the PC is not booted.
This management interface has a weak setup password and must be changed.
The IP on mine is set to static at 192.168.1.100 and defaults to admin/password.
Regardless you should check out the BMC network settings in the BIOS. See what it shows.
The problem is you need to access the actual BMC via a Web interface to change the password.
Seems ridiculous on a server board. Should be in the bios.
 
C2xxx chips dying.... jesus.... the board was kinda expensive..... warranty is over now :(

The BMC has a dedicated LAN port where nothing is plugged in - I never used it. Do the things you've said still aply?
 
I was hoping somebody more network wise would chip in to help. I was just throwing suggestions up. Board updates cannot hurt.
On mine igb0 shares with management.

Just to be clear you are running the isc dhcp server dhcpd right? Have you considered trying dns/dnsmasq?
It is a relatively easy replacement that serves the same function.
 
I guess the question is what is on igb0. Is this your WAN connection(modem)? I assume this board has multiple ethernet interfaces?
Do you have anything special on igb0 like VPN tunnels or Virtual IP(LAN)?
 
Last edited:
OK, BIOS update is done. IPMI Update can not be made as the BMC does not respond. IPMI is also shown as "Not working" in the BIOS. Something seems to be broken with the expensive board. I thought buying supermicro would help me surviving with it for ages :(

On igb0 is my isolated TV which off most of its time, on igb1 is my LAN, on igb2 is my WLAN-AP and on igb3 is my WAN uplink.
igb0 itself has aliases but it is all kinda standard - nothing special I'd say...

Code: 
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:02:03:20
        inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
        inet6 fe80::ec4:7aff:fe02:320%igb0 prefixlen 64 scopeid 0x1
        inet6 2001:db8:2::1 prefixlen 64
        inet 10.0.0.11 netmask 0xffffffff broadcast 10.0.0.255
        inet 10.0.0.12 netmask 0xffffffff broadcast 10.0.0.255
        inet 10.0.0.13 netmask 0xffffffff broadcast 10.0.0.255
        inet 10.0.0.14 netmask 0xffffffff broadcast 10.0.0.255
        inet 10.0.0.15 netmask 0xffffffff broadcast 10.0.0.255
        inet 10.0.0.16 netmask 0xffffffff broadcast 10.0.0.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:02:03:21
        inet 10.0.1.1 netmask 0xffffff00 broadcast 10.0.1.255
        inet6 fe80::ec4:7aff:fe02:321%igb1 prefixlen 64 scopeid 0x2
        inet6 2001:db8::1 prefixlen 64
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:02:03:22
        inet 10.0.2.1 netmask 0xffffff00 broadcast 10.0.2.255
        inet6 fe80::ec4:7aff:fe02:322%igb2 prefixlen 64 scopeid 0x3
        inet6 2001:db8:3::1 prefixlen 64
        inet 10.0.2.14 netmask 0xffffffff broadcast 10.0.2.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
igb3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1400
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:02:03:23
        inet6 fe80::ec4:7aff:fe02:323%igb3 prefixlen 64 scopeid 0x4
        inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1390
        options=80000<LINKSTATE>
        inet6 fe80::ec4:7aff:fe02:320%tun0 prefixlen 64 scopeid 0x6
        inet 10.1.0.10 --> 10.1.0.1 netmask 0xffffffff
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 1075

I run a VPN over igb3. My ipfw rules are:

Code: 
01000 allow ip6 from any to any
02000 count ip from any to any // ==== igb0 igb1 igb2 lo0 ============= allow everything going through LAN
02002 allow ip from me to me via lo0
02004 allow ip from 10.0.0.0/8 to 10.0.0.0/8 { recv igb0 or recv igb1 or recv igb2 } { xmit igb0 or xmit igb1 or xmit igb2 }
02005 allow udp from 0.0.0.0 to 255.255.255.255 dst-port 67 { recv igb0 or recv igb1 or recv igb2 or recv } }
02006 deny log ip from any to any { recv igb0 or recv igb1 or recv igb2 } { xmit igb0 or xmit igb1 or xmit igb2 }
02008 allow ip from 10.0.0.0/8 to any in { via igb0 or via igb1 or via igb2 }
02010 allow ip from any to 10.0.0.0/8 out { via igb0 or via igb1 or via igb2 }
02012 allow udp from me to 239.255.255.250 dst-port 1900 out { via igb0 or via igb1 or via igb2 }
02016 allow igmp from any to 224.0.0.22 out via igb0
02018 deny log ip from any to any { via igb0 or via igb1 or via igb2 }
06000 count ip from any to any // ==== igb3 ====================== deny bad stuff
06002 skipto 6006 ip from any to { me or 10.0.0.0/22 } in via igb3
06003 skipto 6006 ip from any to any dst-port 67-68 in via igb3
06004 deny log ip from any to any in via igb3
06006 deny ip from me to any in via igb3
06008 deny tcp from any to any in via igb3 tcpflags syn,fin
06010 deny ip from 127.0.0.0/8 to any via igb3
06012 deny tcp from any to any dst-port 135,445 in via igb3
06014 deny udp from any to any dst-port 135-138,445,520,694 in via igb3
11940 count ip from any to any // ==== igb3 ====================== filter internet traffic
12000 divert 8668 ip from any to any in via igb3
12010 check-state
12040 skipto 13000 udp from any to any out via igb3 keep-state
12045 skipto 13000 tcp from any to any out via igb3 setup keep-state
12046 skipto 13000 icmp from any to any out via igb3 keep-state
12050 skipto 13000 tcp from any to 10.0.0.13 dst-port 80 in via igb3 setup keep-state
12999 deny log ip from any to any in via igb3
13000 divert 8668 ip from any to any out via igb3
13010 allow ip from any to any via igb3
14000 count ip from any to any // ==== tun0 ====================== VPN traffic
14004 allow ip from any to any via tun0
60000 count ip from any to any // ================================ log+deny everything
60002 deny log ip from any to any
65535 deny ip from any to any


In the past I had on bootup:

Code: 
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi

now when I load the ipmi kernel module I get:

Code: 
ipmi0: <IPMI System Interface> on isa0
ipmi0: KCS mode found at io 0xca2 alignment 0x1 on isa
ipmi0: couldn't find it
device_attach: ipmi0 attach returned 6
ppc0: cannot reserve I/O port range

no idea why isa and no idea why the parallel port is involved... The ppc0 message was always there. Even 2 years ago.
 
You must log in or register to reply here.
Back
Top