FreeBSD move to Proxmox, NICs sometimes can not work

mfaridi

mfaridi

I have FreeBSD physical server. This server work for me 5 years, after 5 years, mainboard is fail and sometimes it can not turn ON, so I move this server to VM by Proxmox, everything is OK and my FreeBSD boot good.
My FreeBSD has two NIC, one NIC with public IP and one NIC with private IP.
I do not know why sometime one NIC can not work good. For example, yesterday when I try to ssh to my server by 192.168.0.210, I can not ssh, so I ssh to server with public IP and I ssh to server by 8.8.8.210, after ssh when I run ifconfig, I see em0 do not have IP, my rc.conf is correct, after reboot everything is OK and I can shh by 192.168.0.210 and 8.8.8.210, but today I can not shh by 8.8.8.210, ifconfig show me em1 has IP, but I can not connect to server.
How I can test what is problem?
 
This is my rc.conf:

Code: 
[root@shialocal ~]# cat /etc/rc.conf
hostname=mfaridi.net
# -- sysinstall generated deltas -- # Thu Feb 14 18:44:57 2008
# Created: Thu Feb 14 18:44:57 2008
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
blanktime="300"
saver="daemon"

defaultrouter="8.8.8.1"
defaultrouter="192.168.0.91"

ifconfig_em0="inet 8.8.8.15   netmask 255.255.255.0"
ifconfig_em1="inet 192.168.0.19  netmask 255.255.255.0"

inetd_enable="YES"
saver="daemon"
sshd_enable="YES"
usbd_enable="YES"

local_startup="/usr/local/etc/rc.d"

pf_enable="YES"                 # Enable PF (load module if required)
pf_rules="/etc/pf.conf"         # rules definition file for PF
pf_flags=""                     # additional flags for pfctl startup
pflog_enable="YES"              # start pflogd(8)
pflog_logfile="/var/log/pflog"  # where pflogd should store the logfile
pflog_flags=""                  # additional flags for pflogd startup


ifconfig_em1_alias0="inet 192.168.0.12 netmask 255.255.255.255"
ifconfig_em1_alias1="inet 192.168.0.13 netmask 255.255.255.255"
ifconfig_em1_alias2="inet 192.168.0.15 netmask 255.255.255.255"
ifconfig_em1_alias3="inet 192.168.0.16 netmask 255.255.255.255"
ifconfig_em1_alias4="inet 192.168.0.17 netmask 255.255.255.255"
ifconfig_em1_alias5="inet 192.168.0.18 netmask 255.255.255.255"
ifconfig_em1_alias6="inet 192.168.0.19 netmask 255.255.255.0"
ifconfig_em1_alias7="inet 192.168.0.20 netmask 255.255.255.255"
ifconfig_em1_alias8="inet 192.168.0.100 netmask 255.255.255.255"

When I run ifconfig -a I see this:
Code: 
[root@shialocal ~]# ifconfig -a
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
    inet 8.8.8.15 netmask 0xffffff00 broadcast 8.8.8.255
    ether 5a:b5:f1:74:77:9d
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
    inet 192.168.0.19 netmask 0xffffff00 broadcast 192.168.0.255
    inet 192.168.0.12 netmask 0xffffffff broadcast 192.168.0.12
    inet 192.168.0.13 netmask 0xffffffff broadcast 192.168.0.13
    inet 192.168.0.15 netmask 0xffffffff broadcast 192.168.0.15
    inet 192.168.0.16 netmask 0xffffffff broadcast 192.168.0.16
    inet 192.168.0.17 netmask 0xffffffff broadcast 192.168.0.17
    inet 192.168.0.18 netmask 0xffffffff broadcast 192.168.0.18
    inet 192.168.0.20 netmask 0xffffffff broadcast 192.168.0.20
    inet 192.168.0.100 netmask 0xffffffff broadcast 192.168.0.100
    ether 3a:ed:a6:3d:b0:a8
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
pfsync0: flags=0<> mtu 2020
    syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33208
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
    inet6 ::1 prefixlen 128
    inet 127.0.0.1 netmask 0xff000000
[root@shialocal ~]#

When I run this command /etc/rc.d/netif restart I see this:
Code: 
Stopping network:ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
lo0 em0 em1 pfsync0 pflog0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
    inet6 ::1 prefixlen 128
    inet 127.0.0.1 netmask 0xff000000
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
    inet 8.8.8.15 netmask 0xffffff00 broadcast 8.8.8.255
    ether 5a:b5:f1:74:77:9d
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
    inet 192.168.0.19 netmask 0xffffff00 broadcast 192.168.0.255
    inet 192.168.0.12 netmask 0xffffffff broadcast 192.168.0.12
    inet 192.168.0.13 netmask 0xffffffff broadcast 192.168.0.13
    inet 192.168.0.15 netmask 0xffffffff broadcast 192.168.0.15
    inet 192.168.0.16 netmask 0xffffffff broadcast 192.168.0.16
    inet 192.168.0.17 netmask 0xffffffff broadcast 192.168.0.17
    inet 192.168.0.18 netmask 0xffffffff broadcast 192.168.0.18
    inet 192.168.0.20 netmask 0xffffffff broadcast 192.168.0.20
    inet 192.168.0.100 netmask 0xffffffff broadcast 192.168.0.100
    ether 3a:ed:a6:3d:b0:a8
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
[root@shialocal ~]#
 
You can only have 1 default router listed in rc.conf. You are setting the default gateway for your machine to the private IP of 192.168.0.1. You will not have Internet access with that setup. Remove the second defaultrouter entry.

You are also setting the same IP on the em1 interface twice, which will cause problems. Remove ifconfig_em1_alias6 and renumber all the ifconfig_em1_alias* entries after that line to fix the numbering.

Not really sure why SSH wouldn't be working. Can you post the ifconfig(8) output when it's not working? Also the output of netstat -rn
 
When I run netstat -rn
I see
Code: 
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
8.8.8/24       link#1             UC          0        0    re0
8.8.8.200      00:19:bb:ce:a8:60  UHLW        1       23    re0   1103
8.8.8.255      ff:ff:ff:ff:ff:ff  UHLWb       1        8    re0
127.0.0.1          127.0.0.1          UH          0        0    lo0
192.168.0          link#2             UC          0        0    em0
192.168.0.12/32    link#2             UC          0        0    em0
192.168.0.13/32    link#2             UC          0        0    em0
192.168.0.15/32    link#2             UC          0        0    em0
192.168.0.16/32    link#2             UC          0        0    em0
192.168.0.17/32    link#2             UC          0        0    em0
192.168.0.18/32    link#2             UC          0        0    em0
192.168.0.20/32    link#2             UC          0        0    em0
192.168.0.100/32   link#2             UC          0        0    em0
192.168.0.232      00:0c:29:07:6b:94  UHLW        1        1    em0   1191

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       link#5                        UHL         lo0
ff01:5::/32                       fe80::1%lo0                   UC          lo0
ff02::%lo0/32                     fe80::1%lo0                   UC          lo0
[root@shialocal ~]#
 
I installed plesk on this server 5 years ago, and this rc.conf used for 5 years, after migration from physical server to VM, I see this error.
After migration when I type https://8.8.8.15:8443 in Firefox, I can not see plesk login pages and I can not login. when I change my IP to public IP like 8.8.8.200, I can see plesk login page and I can use it.
plesk login page only come with 8.8.8 IP and private IP and can not load by other IP.
 
Your rc.conf file shows em1 but your netstat(1) output shows re0. Sounds like you need to update your rc.conf to use the correct interfaces.

In fact, your ifconfig(8) output doesn't match your netstat(1) output (em0/em1 vs re0/em0).

Your netstat(1) output also doesn't list a default router (which would list default in the first column).
 
phoenix said:
Your rc.conf file shows em1 but your netstat(1) output shows re0. Sounds like you need to update your rc.conf to use the correct interfaces.

In fact, your ifconfig(8) output doesn't match your netstat(1) output (em0/em1 vs re0/em0).

Your netstat(1) output also doesn't list a default router (which would list default in the first column).
Click to expand...
Thanks , I change one Lan card and and this is my new netstat -rn
Code: 
[root@shialocal ~]# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            80.191.91.1        UGS         0     8576    re0
8.8.8/24       link#1             UC          0        0    re0
8.8.8.1        00:1c:b1:ed:31:29  UHLW        2        0    re0   1199
8.8.8.30       00:21:5a:bc:f2:99  UHLW        1        5    re0   1000
8.8.8.209      fe:c7:a2:17:73:eb  UHLW        1        4    re0     96
8.8.8.255      ff:ff:ff:ff:ff:ff  UHLWb       1     4527    re0
127.0.0.1          127.0.0.1          UH          0      151    lo0
192.168.0          link#2             UC          0        0    em0
192.168.0.12/32    link#2             UC          0        0    em0
192.168.0.13/32    link#2             UC          0        0    em0
192.168.0.15/32    link#2             UC          0        0    em0
192.168.0.16/32    link#2             UC          0        0    em0
192.168.0.17/32    link#2             UC          0        0    em0
192.168.0.18/32    link#2             UC          0        0    em0
192.168.0.20/32    link#2             UC          0        0    em0
192.168.0.21       00:19:bb:ce:82:6a  UHLW        1       21    em0    351
192.168.0.22       00:19:bb:ce:82:6a  UHLW        1        1    em0   1000
192.168.0.32       08:60:6e:6b:96:c0  UHLW        1       43    em0    882
192.168.0.35       74:d4:35:82:68:96  UHLW        1       27    em0   1116
192.168.0.45       cc:5d:4e:39:ac:6a  UHLW        1       36    em0   1104
192.168.0.56       08:60:6e:6b:96:bf  UHLW        1        8    em0   1174
192.168.0.73       08:60:6e:87:d6:4e  UHLW        1       51    em0   1190
192.168.0.99       00:1a:92:6f:31:36  UHLW        1       10    em0   1194
192.168.0.100/32   link#2             UC          0        0    em0
192.168.0.200      00:21:5a:bc:f2:75  UHLW        1       55    em0   1007
192.168.0.232      00:0c:29:07:6b:94  UHLW        1       15    em0    351

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       link#5                        UHL         lo0
ff01:5::/32                       fe80::1%lo0                   UC          lo0
ff02::%lo0/32                     fe80::1%lo0                   UC          lo0
[
I also edit rc.conf and remove one IP from alias , this is my new rc.conf

Code: 
# This file now contains just the overrides from /etc/defaults/rc.conf.
blanktime="300"
saver="daemon"

defaultrouter="8.8.8.1"
#defaultrouter="192.168.0.91"

ifconfig_re0="inet 8.8.8.15   netmask 255.255.255.0"
ifconfig_em0="inet 192.168.0.19  netmask 255.255.255.0"

inetd_enable="YES"
saver="daemon"
sshd_enable="YES"
usbd_enable="YES"

local_startup="/usr/local/etc/rc.d"

pf_enable="YES"                 # Enable PF (load module if required)
pf_rules="/etc/pf.conf"         # rules definition file for PF
pf_flags=""                     # additional flags for pfctl startup
pflog_enable="YES"              # start pflogd(8)
pflog_logfile="/var/log/pflog"  # where pflogd should store the logfile
pflog_flags=""                  # additional flags for pflogd startup


ifconfig_em0_alias0="inet 192.168.0.12 netmask 255.255.255.255"
ifconfig_em0_alias1="inet 192.168.0.13 netmask 255.255.255.255"
ifconfig_em0_alias2="inet 192.168.0.15 netmask 255.255.255.255"
ifconfig_em0_alias3="inet 192.168.0.16 netmask 255.255.255.255"
ifconfig_em0_alias4="inet 192.168.0.17 netmask 255.255.255.255"
ifconfig_em0_alias5="inet 192.168.0.18 netmask 255.255.255.255"
ifconfig_em0_alias6="inet 192.168.0.20 netmask 255.255.255.255"
ifconfig_em0_alias7="inet 192.168.0.100 netmask 255.255.255.255"
 
I need your advice
We have one server in US and that server is our main server with plesk. This server 8.8.8.15 is local server, we use this server for update site , user in work place update site in Local server and every night local server and main server sync together , I made local server because when we loose internet connection . user can update site.
So I put two default route for server 8.8.8.1 and 192.168.01 , if my local server want sync with main server . it need default route , and user at work place new default route to connect local server.
So what I must do ? I think I have to has two default route .
 
It depends. If "lost internet connection" means, that you lost link on the interface to the internet, you can probably use route metric, see ifconfig(8). If it means that problem would be somewhere upstream, you will probably have to create some script, which will periodically check availability of the remote site and when this one is unavailable then steer traffic according your needs. You can inject new entry to the local DNS, but it is unreliable, because users may have different servers set on their stations and there is always some timeout before locally cached entries expire or you can use pf redirect to the local IP address. Don't forget to return to the "usual" configuration when check "says" remote is up again. You can not change route to the local server, because checking script would "see" that remote end is up again. However you can get around this using multiple routing tables, see setfib(1).
 
ondra_knezour said:
It depends. If "lost internet connection" means, that you lost link on the interface to the internet, you can probably use route metric, see ifconfig(8). If it means that problem would be somewhere upstream, you will probably have to create some script, which will periodically check availability of the remote site and when this one is unavailable then steer traffic according your needs. You can inject new entry to the local DNS, but it is unreliable, because users may have different servers set on their stations and there is always some timeout before locally cached entries expire or you can use pf redirect to the local IP address. Don't forget to return to the "usual" configuration when check "says" remote is up again. You can not change route to the local server, because checking script would "see" that remote end is up again. However you can get around this using multiple routing tables, see setfib(1).
Click to expand...
Thanks.
I can not use setfib, because I use FreeBSD 6.3 and can not upgrade, because plesk do not support FreeBSD 7.
 
You must log in or register to reply here.
Back
Top