FreeBSD shell extremely slow response

Hello,

I'd like to get some ideas over this, from more experienced users because I have no idea how to approach this problem.

I have an ebox 3350mx device, which runs as server on my home network. Runs services like: dnsmasq, afpd, sshd, torrents, tor, etc.

The device seems an excellent choice for small home server, since it's fanless and (in theory) should run most tasks smoothly.

The installed OS is FreeBSD 9.0 stable.

These are the system's specs (got them using 'sysinfo' script from ports):
Code:
[root@comodino ~]# sysinfo -a
Generated by SysInfo v1.0.1 by Daniel Gerzo

System information


INFO: Run `dmidecode -t system` to see further information.

Base board information

INFO: Run `dmidecode -t baseboard` to see further information.

Graphic card information:
vendor='RDC Semiconductor, Inc.'
INFO: Check pciconf(8) for more information.

BIOS information


INFO: Run `dmidecode -t bios` to see further information.

CPU information

Machine class:	i386
CPU Model:	Pentium
No. of Cores:	1
Cores per CPU:	

CPU usage statistics:
CPU:  0.0% user,  0.0% nice,  2.6% system,  0.0% interrupt, 97.4% idle

RAM information

Memory information from dmidecode(8)

INFO: Run `dmidecode -t memory` to see further information.

System memory summary
Total real memory available:	495 MB
Logically used memory:		111 MB
Logically available memory:	384 MB

Swap information
Device          1K-blocks     Used    Avail Capacity
/dev/ada0p3        788480     784k     769M     0%

Operating system information

Operating system release:	FreeBSD 9.0-RELEASE
OS architecture:		i386
Hostname:			comodino
Kernel build dir location:	/usr/obj/usr/src/sys/GENERIC
Currently booted kernel:	/boot/kernel/kernel

Currently loaded kernel modules (kldstat(8)):
ext2fs.ko
zfs.ko
opensolaris.ko
pf.ko
pflog.ko

INFO: For security recommendations see the security(7) man page.
INFO: For system tuning advice, see the tuning(7) man page.
INFO: To view various system statistics use the systat(1) tool.

Bootloader settings
System uptime:
System is up since Thu Sep 6 16:15:51 2012
 1:11PM  up 1 day, 20:55, 1 user, load averages: 0.15, 0.03, 0.01

Process statistics:
33 processes:  1 running, 32 sleeping

Jail related information:
   JID  IP Address      Hostname                      Path

Storage information

Available hard drives:
ada0: <SD RDC SD-IDE HOST CONTROLLER 01000000> ATA-6 device
ada0: 33.300MB/s transfers (UDMA2, PIO 512bytes)
ada0: 15399MB (31537152 512 byte sectors: 16H 63S/T 31286C)
ada0: Previously was known as ad4

Raid controllers:

Currently mounted filesystems:
/dev/ada0p2 on /
devfs on /dev
/dev/da0s1a on /mnt/storage
/dev/da1s1a on /mnt/backup



I/O statistics:
       tty            ada0              da0              da1             cpu
 tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   0    16 11.16   4  0.05  20.29  40  0.79  19.54  77  1.47   7  0 10  2 81
INFO: Run iostat(8) or gstat(8) to see live statistics.

Disk usage:
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ada0p2     13G    3.6G    9.1G    28%    /
devfs          1.0k    1.0k      0B   100%    /dev
/dev/da0s1a    1.8T     20G    1.6T     1%    /mnt/storage
/dev/da1s1a    1.8T     24G    1.6T     1%    /mnt/backup

Network information

hostname: comodino

Currently available network devices:
vte0 lo0 pflog0

Basic configuration for currently available NICs:

vte0 (): status: active
MAC address: 
00:1b:eb:25:1e:3c
IPv4 addresses:
10.0.0.4 netmask 0xffffff00
IPv6 addresses:
fe80::21b:ebff:fe25:1e3c%vte0 prefixlen 64

Input errors: 0
Output errors: 1981346
Collisions: 2495787988

The problem is that it's really slow lately. At the beginning it was running 'fine', slowing down the prompt results every now and then, but lately is utterly slow. When I type 'uname -a' sometimes I have to wait 20 seconds to get a result. Some times 'vim' opens a file as I hit enter and other times I have to wait 40 seconds.

In many situations 'uptime' show a load-avg of 0.00% ... Sometimes shows a load avg of 0.25% but the cli runs much more smoothly.

Things get worst - meaning unbearable - since my provider's problem got me offline the last couple of days. The only error I have seen in the system so far is sendmail related:

Code:
Sep  8 04:28:44 comodino sendmail[10543]: unable to qualify my own domain name (comodino) -- using short name
Sep  8 04:28:44 comodino sendmail[10536]: unable to qualify my own domain name (comodino) -- using short name

I don't know why, it keeps looking for 'qualified domain name'. I'll add a dyndns address for OpenVPN services. I don't know how sendmail works and how to fix this. I use postfix usually, but didn't get there yet.

Is there any way to make this thing go faster that you can think of?

Thanks for you time
 
It sounds like DNS. That should not make responses to the command line slow unless you have something like a custom prompt that needs DNS resolution. Make certain that /etc/hosts has a 127.0.0.1 entry for localhost and the actual hostname. If DNS is not working, there will be long timeouts as programs try and fail to resolve hostnames.

If the machine is severely RAM-starved, it could be swapping. The output of top(1) will show that. If swapping is the problem, it should be fast after a reboot.

Whatever is being used for a hard drive could be failing.
 
Hello,

Thanks for the prompt reply. Here is the output of top:

Code:
last pid: 14113;  load averages:  0.00,  0.00,  0.00                                            up 2+01:19:12  17:35:03
24 processes:  1 running, 23 sleeping
CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
Mem: 25M Active, 341M Inact, 82M Wired, 24M Cache, 59M Buf, 14M Free
Swap: 770M Total, 816K Used, 769M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
14111 atma          1  20    0  9944K  1644K RUN      0:00  0.20% top
 8329 root          5  20    0 29004K  6632K select   1:41  0.00% python2.7
 7680 nobody        1  20    0 11180K  1508K select   0:15  0.00% dnsmasq
 1181 root          1  20    0 11324K  1956K select   0:08  0.00% sendmail
 1202 root          1  20    0  9644K   500K nanslp   0:02  0.00% cron
  878 root          1  20    0  9612K   916K select   0:01  0.00% syslogd
 1157 root          1  20    0 13064K  1828K select   0:01  0.00% sshd
14106 root          1  20    0 15788K  3776K sbwait   0:00  0.00% sshd
 1196 smmsp         1  20    0 11324K   760K pause    0:00  0.00% sendmail
  742 root          1  20    0 12128K  2332K select   0:00  0.00% devd
 6917 root          1  24    0 10948K     0K pause    0:00  0.00% <csh>
 6916 root          1  20    0 10268K  1488K select   0:00  0.00% screen
14109 atma          1  20    0 10832K  2288K wait     0:00  0.00% bash
 6921 root          1  20    0 10832K  1356K ttyin    0:00  0.00% bash
14108 atma          1  20    0 15788K  3772K select   0:00  0.00% sshd
 1264 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1265 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1266 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1270 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1269 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1268 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1271 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 1267 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
 107 root          1  52    0  9532K     0K pause    0:00  0.00% <adjkerntz>

Uses most of it's build in mem with only 14 mb free and a load of 0.00%.

Actually the ebox started acting *that slow* when my modem started disconnecting every 30 minutes. Then it turned out worst, I'm expecting a tech guy to fix the cables on Monday morning.

I will wait till then to see what happens. I will configure dyndns and try to "fix" sendmail to act normally and not displaying any messages and see what happens.

I will post an updated.

Thanks
 
You don't need dyndns for fixing the sendmail(8) issue, all you need is an IP address in /etc/hosts that matches the hostname you have set in /etc/rc.conf.


If you have in /etc/rc.conf:
Code:
hostname="myhost.mydomain"

Add this to /etc/hosts. 192.168.1.10 is just an example here, replace with an address that is statically configured on any interface on the machine.

Code:
192.168.1.10  myhost.mydomain myhost

Or if all you have is one interface that is configure dynamically with DHCP you can use the localhost address 127.0.0.1. Add this to /etc/hosts, don't replace the localhost line:

Code:
127.0.0.1 myhost.mydomain myhost
 
kpa said:
You don't need dyndns for fixing the sendmail(8) issue, all you need is an IP address in /etc/hosts that matches the hostname you have set in /etc/rc.conf.


If you have in /etc/rc.conf:
Code:
hostname="myhost.mydomain"

Add this to /etc/hosts. 192.168.1.10 is just an example here, replace with an address that is statically configured on any interface on the machine.

Code:
192.168.1.10  myhost.mydomain myhost

Or if all you have is one interface that is configure dynamically with DHCP you can use the localhost address 127.0.0.1. Add this to /etc/hosts, don't replace the localhost line:

Code:
127.0.0.1 myhost.mydomain myhost

Thanks for the hint. I thought it needed a full qualified to domain name to sent receive emails. I adjusted my configuration accordingly.
 
It's the default one. I didn't even bothered since .zsh + oh-my-zsh was too slow, I just stuck with bash:

Code:
[user@comodino ~]$

that's it.
 
The thinking is that it's trying to resolve the hostname each time, and failing. Before changing anything, what does this show?
% nslookup comodino
 
wblock@ said:
The thinking is that it's trying to resolve the hostname each time, and failing. Before changing anything, what does this show?
% nslookup comodino

As I mentioned my ISP will fix my connection monday, so this machine is currently offline.

Code:
[user@comodino ~]$ time nslookup comodino
;; connection timed out; no servers could be reached

real	0m21.060s
user	0m0.035s
sys	0m0.018s
 
All right. Does it start reacting quickly with the hostname added to /etc/hosts?

Code:
127.0.0.1       localhost comodino
 
Hello,

it's a bit faster, but not 'smoth' at all.


Code:
[user@comodino ~]$ time nslookup comodino
;; connection timed out; no servers could be reached


real	0m21.107s
user	0m0.021s
sys	0m0.037s

Could be the SD card?

Code:
[user@comodino ~]$ iostat
       tty            ada0              da0              da1             cpu
 tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   0    12 10.92   3  0.04  19.75  28  0.53  18.87  54  1.00   5  0  7  2 87

I don't have access to another computer to have a look and get a clue about avg IDE/SATA statistics. I'm on a macbook air ssd, but values are not comparable, neither avg operation speed.
 
Adding some more feedback. I shutdown the machine and booted from the SD card on a VirtualBox. On the VirtualBox with only 128 MB memory, bash runs normally, it's as responsive as it gets. However in some situations seem to run slow on the VB also, namely:

1) The login prompt, takes a considerable amount of time after I issue user and password. After username takes 10-15 seconds which explains why when I'm connecting via ssh it's soo slow. I wonder if PAM has something to do with this...

2) nslookup is as slow as in the ebox, although makes sense. In ebox the iface is vte0 (and it's configured) in the VB it's em0 and it's not so…

3) vim: The first time after every boot in the VB is damn slow. It's slow until it open's the file. Although I have vim-swap disable (along with another option to improve performance which I can't recall now).

Now I stopped all services except dnsmasq. This is my rc.conf:
Code:
# System
hostname="comodino.karida.ath.cx"
ifconfig_vte0="inet 10.0.0.4 netmask 255.255.255.0"
defaultrouter="10.0.0.138"
#ifconfig_vte0="DHCP"

# Packet Filter enable
pf_enable="NO"
pf_rules="/etc/pf.conf"
pflog_enable="NO"
pflog_logfile="/var/log/pflog"
accounting_enable="NO"

# System Services
sshd_enable="YES"
dnsmasq_enable="YES"
ntpd_enable="NO"
ntpdate_enable="NO"
ntpdate_hosts="ntp.forthnet.gr"
fail2ban_enable="NO"

# Netatalk configuration
netatalk_enable="NO"
cnid_metad_enable="NO"
afpd_enable="NO"
dbus_enable="NO"
avahi_daemon_enable="NO"
mongod_enable="NO"

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable

I have the strong feeling that this is a network related issue at some level by the following test I've made. I ping'ed all hosts in our network which are: a) wireless hp-server (10.0.0.1), gateway (10.0.0.138) and comodino (the freebsd ebox) 10.0.0.4 . Comodino as expected performed poorly compared to others although it's severely over-powered in comparison:
Code:
➜  ~  ping -c 50 -q 10.0.0.4     
PING 10.0.0.4 (10.0.0.4): 56 data bytes

--- 10.0.0.4 ping statistics ---
50 packets transmitted, 50 packets received, 0.0% packet loss
round-trip min/[B]avg[/B]/max/stddev = 3.324/[B]223.291[/B]/950.957/251.607 ms

➜  chap2  ping -c 50 -q 10.0.0.4    ([B]pf / pflogd disabled[/B])
PING 10.0.0.4 (10.0.0.4): 56 data bytes

--- 10.0.0.4 ping statistics ---
50 packets transmitted, 50 packets received, 0.0% packet loss
round-trip min/[B]avg[/B]/max/stddev = 1.048/[B]252.674[/B]/687.965/201.663 ms

➜  ~  ping -c 50 -q 10.0.0.1 (hp wifi printer)
PING 10.0.0.1 (10.0.0.1): 56 data bytes

--- 10.0.0.1 ping statistics ---
50 packets transmitted, 50 packets received, 0.0% packet loss
round-trip min/[B]avg[/B]/max/stddev = 3.508/[B]37.959[/B]/269.053/51.356 ms

➜  ~  ping -c 50 -q 10.0.0.138 (zyxel router)
PING 10.0.0.138 (10.0.0.138): 56 data bytes

--- 10.0.0.138 ping statistics ---
50 packets transmitted, 50 packets received, 0.0% packet loss
round-trip min/[B]avg[/B]/max/stddev = 1.149/[B]16.276[/B]/193.699/26.726 ms
➜  ~

nmap scan's sometimes do not even reveal the host in a ping sweep because - I think - it delays too much the reply, or doesn't reply at all(?).
Code:
nmap scans 
➜  ~  nmap -sP 10.0.0.1/24     

Starting Nmap 6.01 ( http://nmap.org ) at 2012-09-09 12:26 CEST
Strange error from connect (65):No route to host
Nmap scan report for hp.karida.ath.cx (10.0.0.1)
Host is up (0.012s latency).
Nmap scan report for greyjewel.karida.ath.cx (10.0.0.2)
Host is up (0.0024s latency).
[B]Nmap scan report for comodino.karida.ath.cx (10.0.0.4)
Host is up (0.013s latency).[/B]
Nmap scan report for gw.karida.ath.cx (10.0.0.138)
Host is up (0.013s latency).
Nmap done: 256 IP addresses (4 hosts up) scanned in 1.52 seconds
➜  ~  nmap -sP 10.0.0.1/24

Starting Nmap 6.01 ( http://nmap.org ) at 2012-09-09 12:26 CEST
Strange error from connect (64):Host is down
Nmap scan report for hp.karida.ath.cx (10.0.0.1)
Host is up (0.043s latency).
Nmap scan report for greyjewel.karida.ath.cx (10.0.0.2)
Host is up (0.0024s latency).
Nmap scan report for gw.karida.ath.cx (10.0.0.138)
Host is up (0.048s latency).
Nmap done: 256 IP addresses (3 hosts up) scanned in 1.98 seconds
➜  ~  nmap -sP 10.0.0.1/24

Starting Nmap 6.01 ( http://nmap.org ) at 2012-09-09 12:26 CEST
Nmap scan report for hp.karida.ath.cx (10.0.0.1)
Host is up (0.056s latency).
Nmap scan report for greyjewel.karida.ath.cx (10.0.0.2)
Host is up (0.00056s latency).
Nmap scan report for gw.karida.ath.cx (10.0.0.138)
Host is up (0.14s latency).
Nmap done: 256 IP addresses (3 hosts up) scanned in 3.51 seconds
➜  ~

in this example the latency is near 'hp' but in several tests comodino's latency (when appeared) was something 0.15 or 0.25 which is way higher then others.

Thanks a priori for the time you took to read this :)

UPDATE [1]
follow-up: I disabled PAM in sshd, however now the login time is 'acceptable' (about 5 seconds) with PAM and 2 seconds without PAM. Working on bash prompt is still cluttered, not extremely slow like before but not smooth (at all) either. Just a lot faster than before.

UPDATE[2]
Apparently what made the diff on ssh-login was the "useDNS no" switch. I disabled sendmail which released 90% of memory, now I have 480+ MB Free but still bash is slow.

Some times I have to press 'enter' twice to get the results of a command on my screen.
 
When the internet connection were back up by mounting the / partition with 'noatime' options everything runs smoothly now :)
 
Back
Top