1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

FreeBSD shell extremely slow response

Discussion in 'General' started by atmosx, Sep 8, 2012.

  1. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    Hello,

    I'd like to get some ideas over this, from more experienced users because I have no idea how to approach this problem.

    I have an ebox 3350mx device, which runs as server on my home network. Runs services like: dnsmasq, afpd, sshd, torrents, tor, etc.

    The device seems an excellent choice for small home server, since it's fanless and (in theory) should run most tasks smoothly.

    The installed OS is FreeBSD 9.0 stable.

    These are the system's specs (got them using 'sysinfo' script from ports):
    Code:
    [root@comodino ~]# sysinfo -a
    Generated by SysInfo v1.0.1 by Daniel Gerzo
    
    System information
    
    
    INFO: Run `dmidecode -t system` to see further information.
    
    Base board information
    
    INFO: Run `dmidecode -t baseboard` to see further information.
    
    Graphic card information:
    vendor='RDC Semiconductor, Inc.'
    INFO: Check pciconf(8) for more information.
    
    BIOS information
    
    
    INFO: Run `dmidecode -t bios` to see further information.
    
    CPU information
    
    Machine class:	i386
    CPU Model:	Pentium
    No. of Cores:	1
    Cores per CPU:	
    
    CPU usage statistics:
    CPU:  0.0% user,  0.0% nice,  2.6% system,  0.0% interrupt, 97.4% idle
    
    RAM information
    
    Memory information from dmidecode(8)
    
    INFO: Run `dmidecode -t memory` to see further information.
    
    System memory summary
    Total real memory available:	495 MB
    Logically used memory:		111 MB
    Logically available memory:	384 MB
    
    Swap information
    Device          1K-blocks     Used    Avail Capacity
    /dev/ada0p3        788480     784k     769M     0%
    
    Operating system information
    
    Operating system release:	FreeBSD 9.0-RELEASE
    OS architecture:		i386
    Hostname:			comodino
    Kernel build dir location:	/usr/obj/usr/src/sys/GENERIC
    Currently booted kernel:	/boot/kernel/kernel
    
    Currently loaded kernel modules (kldstat(8)):
    ext2fs.ko
    zfs.ko
    opensolaris.ko
    pf.ko
    pflog.ko
    
    INFO: For security recommendations see the security(7) man page.
    INFO: For system tuning advice, see the tuning(7) man page.
    INFO: To view various system statistics use the systat(1) tool.
    
    Bootloader settings
    System uptime:
    System is up since Thu Sep 6 16:15:51 2012
     1:11PM  up 1 day, 20:55, 1 user, load averages: 0.15, 0.03, 0.01
    
    Process statistics:
    33 processes:  1 running, 32 sleeping
    
    Jail related information:
       JID  IP Address      Hostname                      Path
    
    Storage information
    
    Available hard drives:
    ada0: <SD RDC SD-IDE HOST CONTROLLER 01000000> ATA-6 device
    ada0: 33.300MB/s transfers (UDMA2, PIO 512bytes)
    ada0: 15399MB (31537152 512 byte sectors: 16H 63S/T 31286C)
    ada0: Previously was known as ad4
    
    Raid controllers:
    
    Currently mounted filesystems:
    /dev/ada0p2 on /
    devfs on /dev
    /dev/da0s1a on /mnt/storage
    /dev/da1s1a on /mnt/backup
    
    
    
    I/O statistics:
           tty            ada0              da0              da1             cpu
     tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
       0    16 11.16   4  0.05  20.29  40  0.79  19.54  77  1.47   7  0 10  2 81
    INFO: Run iostat(8) or gstat(8) to see live statistics.
    
    Disk usage:
    Filesystem     Size    Used   Avail Capacity  Mounted on
    /dev/ada0p2     13G    3.6G    9.1G    28%    /
    devfs          1.0k    1.0k      0B   100%    /dev
    /dev/da0s1a    1.8T     20G    1.6T     1%    /mnt/storage
    /dev/da1s1a    1.8T     24G    1.6T     1%    /mnt/backup
    
    Network information
    
    hostname: comodino
    
    Currently available network devices:
    vte0 lo0 pflog0
    
    Basic configuration for currently available NICs:
    
    vte0 (): status: active
    MAC address: 
    00:1b:eb:25:1e:3c
    IPv4 addresses:
    10.0.0.4 netmask 0xffffff00
    IPv6 addresses:
    fe80::21b:ebff:fe25:1e3c%vte0 prefixlen 64
    
    Input errors: 0
    Output errors: 1981346
    Collisions: 2495787988
    


    The problem is that it's really slow lately. At the beginning it was running 'fine', slowing down the prompt results every now and then, but lately is utterly slow. When I type 'uname -a' sometimes I have to wait 20 seconds to get a result. Some times 'vim' opens a file as I hit enter and other times I have to wait 40 seconds.

    In many situations 'uptime' show a load-avg of 0.00% ... Sometimes shows a load avg of 0.25% but the cli runs much more smoothly.

    Things get worst - meaning unbearable - since my provider's problem got me offline the last couple of days. The only error I have seen in the system so far is sendmail related:

    Code:
    Sep  8 04:28:44 comodino sendmail[10543]: unable to qualify my own domain name (comodino) -- using short name
    Sep  8 04:28:44 comodino sendmail[10536]: unable to qualify my own domain name (comodino) -- using short name
    


    I don't know why, it keeps looking for 'qualified domain name'. I'll add a dyndns address for OpenVPN services. I don't know how sendmail works and how to fix this. I use postfix usually, but didn't get there yet.

    Is there any way to make this thing go faster that you can think of?

    Thanks for you time
     
  2. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,693
    Thanks Received:
    2,265
    It sounds like DNS. That should not make responses to the command line slow unless you have something like a custom prompt that needs DNS resolution. Make certain that /etc/hosts has a 127.0.0.1 entry for localhost and the actual hostname. If DNS is not working, there will be long timeouts as programs try and fail to resolve hostnames.

    If the machine is severely RAM-starved, it could be swapping. The output of top(1) will show that. If swapping is the problem, it should be fast after a reboot.

    Whatever is being used for a hard drive could be failing.
     
  3. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    Hello,

    Thanks for the prompt reply. Here is the output of top:

    Code:
    last pid: 14113;  load averages:  0.00,  0.00,  0.00                                            up 2+01:19:12  17:35:03
    24 processes:  1 running, 23 sleeping
    CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
    Mem: 25M Active, 341M Inact, 82M Wired, 24M Cache, 59M Buf, 14M Free
    Swap: 770M Total, 816K Used, 769M Free
    
      PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
    14111 atma          1  20    0  9944K  1644K RUN      0:00  0.20% top
     8329 root          5  20    0 29004K  6632K select   1:41  0.00% python2.7
     7680 nobody        1  20    0 11180K  1508K select   0:15  0.00% dnsmasq
     1181 root          1  20    0 11324K  1956K select   0:08  0.00% sendmail
     1202 root          1  20    0  9644K   500K nanslp   0:02  0.00% cron
      878 root          1  20    0  9612K   916K select   0:01  0.00% syslogd
     1157 root          1  20    0 13064K  1828K select   0:01  0.00% sshd
    14106 root          1  20    0 15788K  3776K sbwait   0:00  0.00% sshd
     1196 smmsp         1  20    0 11324K   760K pause    0:00  0.00% sendmail
      742 root          1  20    0 12128K  2332K select   0:00  0.00% devd
     6917 root          1  24    0 10948K     0K pause    0:00  0.00% <csh>
     6916 root          1  20    0 10268K  1488K select   0:00  0.00% screen
    14109 atma          1  20    0 10832K  2288K wait     0:00  0.00% bash
     6921 root          1  20    0 10832K  1356K ttyin    0:00  0.00% bash
    14108 atma          1  20    0 15788K  3772K select   0:00  0.00% sshd
     1264 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1265 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1266 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1270 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1269 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1268 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1271 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     1267 root          1  52    0  9612K   788K ttyin    0:00  0.00% getty
     107 root          1  52    0  9532K     0K pause    0:00  0.00% <adjkerntz>
    


    Uses most of it's build in mem with only 14 mb free and a load of 0.00%.

    Actually the ebox started acting *that slow* when my modem started disconnecting every 30 minutes. Then it turned out worst, I'm expecting a tech guy to fix the cables on Monday morning.

    I will wait till then to see what happens. I will configure dyndns and try to "fix" sendmail to act normally and not displaying any messages and see what happens.

    I will post an updated.

    Thanks
     
  4. kpa

    kpa Well-Known Member

    Messages:
    4,106
    Thanks Received:
    812
    You don't need dyndns for fixing the sendmail(8) issue, all you need is an IP address in /etc/hosts that matches the hostname you have set in /etc/rc.conf.


    If you have in /etc/rc.conf:
    Code:
    hostname="myhost.mydomain"
    


    Add this to /etc/hosts. 192.168.1.10 is just an example here, replace with an address that is statically configured on any interface on the machine.

    Code:
    192.168.1.10  myhost.mydomain myhost
    


    Or if all you have is one interface that is configure dynamically with DHCP you can use the localhost address 127.0.0.1. Add this to /etc/hosts, don't replace the localhost line:

    Code:
    127.0.0.1 myhost.mydomain myhost
    
     
  5. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    Thanks for the hint. I thought it needed a full qualified to domain name to sent receive emails. I adjusted my configuration accordingly.
     
  6. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,693
    Thanks Received:
    2,265
    What is your bash prompt command?
     
  7. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    It's the default one. I didn't even bothered since .zsh + oh-my-zsh was too slow, I just stuck with bash:

    Code:
    [user@comodino ~]$


    that's it.
     
  8. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,693
    Thanks Received:
    2,265
    The thinking is that it's trying to resolve the hostname each time, and failing. Before changing anything, what does this show?
    % nslookup comodino
     
  9. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    As I mentioned my ISP will fix my connection monday, so this machine is currently offline.

    Code:
    [user@comodino ~]$ time nslookup comodino
    ;; connection timed out; no servers could be reached
    
    real	0m21.060s
    user	0m0.035s
    sys	0m0.018s
    
     
  10. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,693
    Thanks Received:
    2,265
    All right. Does it start reacting quickly with the hostname added to /etc/hosts?

    Code:
    127.0.0.1       localhost comodino
    
     
  11. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    Hello,

    it's a bit faster, but not 'smoth' at all.


    Code:
    [user@comodino ~]$ time nslookup comodino
    ;; connection timed out; no servers could be reached
    
    
    real	0m21.107s
    user	0m0.021s
    sys	0m0.037s
    


    Could be the SD card?

    Code:
    [user@comodino ~]$ iostat
           tty            ada0              da0              da1             cpu
     tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
       0    12 10.92   3  0.04  19.75  28  0.53  18.87  54  1.00   5  0  7  2 87
    


    I don't have access to another computer to have a look and get a clue about avg IDE/SATA statistics. I'm on a macbook air ssd, but values are not comparable, neither avg operation speed.
     
  12. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    Adding some more feedback. I shutdown the machine and booted from the SD card on a VirtualBox. On the VirtualBox with only 128 MB memory, bash runs normally, it's as responsive as it gets. However in some situations seem to run slow on the VB also, namely:

    1) The login prompt, takes a considerable amount of time after I issue user and password. After username takes 10-15 seconds which explains why when I'm connecting via ssh it's soo slow. I wonder if PAM has something to do with this...

    2) nslookup is as slow as in the ebox, although makes sense. In ebox the iface is vte0 (and it's configured) in the VB it's em0 and it's not so…

    3) vim: The first time after every boot in the VB is damn slow. It's slow until it open's the file. Although I have vim-swap disable (along with another option to improve performance which I can't recall now).

    Now I stopped all services except dnsmasq. This is my rc.conf:
    Code:
    # System
    hostname="comodino.karida.ath.cx"
    ifconfig_vte0="inet 10.0.0.4 netmask 255.255.255.0"
    defaultrouter="10.0.0.138"
    #ifconfig_vte0="DHCP"
    
    # Packet Filter enable
    pf_enable="NO"
    pf_rules="/etc/pf.conf"
    pflog_enable="NO"
    pflog_logfile="/var/log/pflog"
    accounting_enable="NO"
    
    # System Services
    sshd_enable="YES"
    dnsmasq_enable="YES"
    ntpd_enable="NO"
    ntpdate_enable="NO"
    ntpdate_hosts="ntp.forthnet.gr"
    fail2ban_enable="NO"
    
    # Netatalk configuration
    netatalk_enable="NO"
    cnid_metad_enable="NO"
    afpd_enable="NO"
    dbus_enable="NO"
    avahi_daemon_enable="NO"
    mongod_enable="NO"
    
    # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
    


    I have the strong feeling that this is a network related issue at some level by the following test I've made. I ping'ed all hosts in our network which are: a) wireless hp-server (10.0.0.1), gateway (10.0.0.138) and comodino (the freebsd ebox) 10.0.0.4 . Comodino as expected performed poorly compared to others although it's severely over-powered in comparison:
    Code:
    ➜  ~  ping -c 50 -q 10.0.0.4     
    PING 10.0.0.4 (10.0.0.4): 56 data bytes
    
    --- 10.0.0.4 ping statistics ---
    50 packets transmitted, 50 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 3.324/223.291/950.957/251.607 ms
    
    ➜  chap2  ping -c 50 -q 10.0.0.4    (pf / pflogd disabled)
    PING 10.0.0.4 (10.0.0.4): 56 data bytes
    
    --- 10.0.0.4 ping statistics ---
    50 packets transmitted, 50 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 1.048/252.674/687.965/201.663 ms
    
    ➜  ~  ping -c 50 -q 10.0.0.1 (hp wifi printer)
    PING 10.0.0.1 (10.0.0.1): 56 data bytes
    
    --- 10.0.0.1 ping statistics ---
    50 packets transmitted, 50 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 3.508/37.959/269.053/51.356 ms
    
    ➜  ~  ping -c 50 -q 10.0.0.138 (zyxel router)
    PING 10.0.0.138 (10.0.0.138): 56 data bytes
    
    --- 10.0.0.138 ping statistics ---
    50 packets transmitted, 50 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 1.149/16.276/193.699/26.726 ms
    ➜  ~  
    


    nmap scan's sometimes do not even reveal the host in a ping sweep because - I think - it delays too much the reply, or doesn't reply at all(?).
    Code:
    nmap scans 
    ➜  ~  nmap -sP 10.0.0.1/24     
    
    Starting Nmap 6.01 ( http://nmap.org ) at 2012-09-09 12:26 CEST
    Strange error from connect (65):No route to host
    Nmap scan report for hp.karida.ath.cx (10.0.0.1)
    Host is up (0.012s latency).
    Nmap scan report for greyjewel.karida.ath.cx (10.0.0.2)
    Host is up (0.0024s latency).
    Nmap scan report for comodino.karida.ath.cx (10.0.0.4)
    Host is up (0.013s latency).
    Nmap scan report for gw.karida.ath.cx (10.0.0.138)
    Host is up (0.013s latency).
    Nmap done: 256 IP addresses (4 hosts up) scanned in 1.52 seconds
    ➜  ~  nmap -sP 10.0.0.1/24
    
    Starting Nmap 6.01 ( http://nmap.org ) at 2012-09-09 12:26 CEST
    Strange error from connect (64):Host is down
    Nmap scan report for hp.karida.ath.cx (10.0.0.1)
    Host is up (0.043s latency).
    Nmap scan report for greyjewel.karida.ath.cx (10.0.0.2)
    Host is up (0.0024s latency).
    Nmap scan report for gw.karida.ath.cx (10.0.0.138)
    Host is up (0.048s latency).
    Nmap done: 256 IP addresses (3 hosts up) scanned in 1.98 seconds
    ➜  ~  nmap -sP 10.0.0.1/24
    
    Starting Nmap 6.01 ( http://nmap.org ) at 2012-09-09 12:26 CEST
    Nmap scan report for hp.karida.ath.cx (10.0.0.1)
    Host is up (0.056s latency).
    Nmap scan report for greyjewel.karida.ath.cx (10.0.0.2)
    Host is up (0.00056s latency).
    Nmap scan report for gw.karida.ath.cx (10.0.0.138)
    Host is up (0.14s latency).
    Nmap done: 256 IP addresses (3 hosts up) scanned in 3.51 seconds
    ➜  ~  
    


    in this example the latency is near 'hp' but in several tests comodino's latency (when appeared) was something 0.15 or 0.25 which is way higher then others.

    Thanks a priori for the time you took to read this :)

    UPDATE [1]
    follow-up: I disabled PAM in sshd, however now the login time is 'acceptable' (about 5 seconds) with PAM and 2 seconds without PAM. Working on bash prompt is still cluttered, not extremely slow like before but not smooth (at all) either. Just a lot faster than before.

    UPDATE[2]
    Apparently what made the diff on ssh-login was the "useDNS no" switch. I disabled sendmail which released 90% of memory, now I have 480+ MB Free but still bash is slow.

    Some times I have to press 'enter' twice to get the results of a command on my screen.
     
  13. atmosx

    atmosx New Member

    Messages:
    68
    Thanks Received:
    3
    When the internet connection were back up by mounting the / partition with 'noatime' options everything runs smoothly now :)