Odd/Slow Terminal Behavior

Code:
FreeBSD <someservername> 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

Yesterday I acquired new hardware (for me anyways) in the form of an HP DL160 G6. I have been running on an old Dell 2950 for the past few years. I decided to do a hard drive transplant instead of a fresh install for simplicity sake. Everything went pretty well.. Had to change /etc/fstab and network card config, but other than that, everything just works..... With the exception of the shell both on the machine itself and via ssh.

The behavior is slow to respond command execution (or display of execution at least) and when I run any ncurses based tools like top or tools like ping that poll. With top for instance it will open up like this:

Code:
last pid:   942;  load averages:  1.66,  1.22,  0.60                                                                                                                                                                                                       up 0+00:16:49  09:23:21
46 processes:  2 running, 43 sleeping, 1 waiting
CPU:     % user,     % nice,     % system,     % interrupt,     % idle
Mem: 84M Active, 33M Inact, 524M Wired, 207M Buf, 23G Free
ARC: 5618K Total, 570K MFU, 4227K MRU, 16K Anon, 131K Header, 673K Other
Swap:

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
  11 root         16 155 ki31     0K   256K CPU15  15 444:49 1589.26% idle
  12 root         47 -84    -     0K   752K WAIT    0   0:00   9.67% intr
    2 root          4 -16    -     0K    64K -       0   0:00   0.00% cam
    0 root        229  -8    0     0K  3664K -       3   0:00   0.00% kernel
  523 bind         19  52    0   111M 36096K kqread  0   0:00   0.00% named
  790 root          1  20    0 26032K  5756K select  5   0:00   0.00% openvpn
  777 puppet        3  20    0   123M 43152K select 12   0:00   0.00% ruby20
  13 root          3  -8    -     0K    48K -       2   0:00   0.00% geom
  691 root          1  52    0 20552K  4816K select 11   0:00   0.00% nfsd
  918 root          1  20    0 86480K  7016K select  9   0:00   0.00% sshd
  15 root         24 -68    -     0K   384K -       0   0:00   0.00% usb
  34 root          5  -8    -     0K    96K tx->tx 13   0:00   0.00% zfskern
    1 root          1  20    0  9472K   872K wait    6   0:00   0.00% init
  14 root          1 -16    -     0K    16K -       9   0:00   0.00% rand_harvestq
  840 root          1  20    0 41308K 10540K nanslp  8   0:00   0.00% perl
  926 root          1  52    0 23580K  2848K pause   4   0:00   0.00% csh
  558 root          1  20    0 14508K  2132K select 11   0:00   0.00% syslogd
  689 root          1  52    0 22700K  4920K select  7   0:00   0.00% mountd
  929 root          1  20    0 17692K  3344K wait   11   0:00   0.00% bash
  920 letme1n       1  20    0 86480K  7112K select 14   0:00   0.00% sshd
  924 root          1  27    0 50296K  3312K select  6   0:00   0.00% sudo
  564 root          1  20    0 16588K  2208K select  6   0:00   0.00% rpcbind
    9 root          1  16    -     0K    16K syncer 15   0:00   0.00% syncer
  921 letme1n       1  27    0 17072K  2652K wait    0   0:00   0.00% sh
  925 root          1  28    0 47712K  2732K wait   10   0:00   0.00% su
    4 root          1 -16    -     0K    16K psleep  7   0:00   0.00% pagedaemon
  692 root        128  52    0 12360K  3936K rpcsvc 14   0:00   0.00% nfsd
  861 root          1  20    0 16600K  2264K nanslp  6   0:00   0.00% cron
  437 root          1  20    0 13164K  4548K select  9   0:00   0.00% devd
  942 root          1  20    0 21924K  2712K CPU6    6   0:00   0.00% top
    7 root          2  20    -     0K    32K sdflus  6   0:00   0.00% bufdaemon
  913 root          1  52    0 14496K  1984K ttyin   8   0:00   0.00% getty
  910 root          1  52    0 14496K  1984K ttyin  12   0:00   0.00% getty
  908 root          1  52    0 14496K  1984K ttyin  14   0:00   0.00% getty
  907 root          1  52    0 14496K  1984K ttyin   5   0:00   0.00% getty
  911 root          1  52    0 14496K  1984K ttyin   7   0:00   0.00% getty
  906 root          1  52    0 14496K  1984K ttyin  11   0:00   0.00% getty
  909 root          1  52    0 14496K  1984K ttyin   0   0:00   0.00% getty
  912 root          1  52    0 14496K  1984K ttyin   2   0:00   0.00% getty
  836 root          1  20    0 61212K  6468K select  3   0:00   0.00% sshd
    8 root          1 -16    -     0K    16K vlruwt  4   0:00   0.00% vnlru
  448 root          1  52    0 16708K  2032K select 14   0:00   0.00% moused
    3 root          1 -16    -     0K    16K waitin 13   0:00   0.00% sctp_iterator
    6 root          1 155 ki31     0K    16K pgzero 12   0:00   0.00% pagezero
    5 root          1 -16    -     0K    16K psleep  7   0:00   0.00% vmdaemon
  10 root          1 -16    -     0K    16K audit_  0   0:00   0.00% audit

and just sit there for 10-20 seconds before it updates.. then it updates at various times inconsistently.

Ping will only return the first line of output and then sit there for 10-20 seconds. This happens at the local terminal and any ssh sessions.

The rest of the system works great (and fast.) For instance I upgraded my ports and the text was screaming by while they built.

I've searched the forums and google for anything similar exhaustively, but haven't found anyone with quite the same problem. Any ideas?

Thanks
 
Maybe you should first try to stop powerd and see if that changes something. The delay to wake a CPU from deeper sleep states can be too high for smooth operation. In that case it helps to set the highest available sleep state for at least one core to something with short latencies to answer interrupts fast enough.
 
I had already disabled that as a first attempt at resolution. The other thing I'm noticing is that the server is losing time in a MAJOR way (as in 6+ hours in a day.) I read up on it and tried switching the timecounter over to i8254 via sysctl, but if anything that's made it worse. I'm starting to wonder if I just have a bad MB/Chip... I have a duplicate of the machine. I'm thinking I'll swap it in to see if it makes a difference.. I may also try a fresh install on an alternate drive. I'm still interested if anyone has more troubleshooting ideas to try. Thanks for your help Crivens.
 
Do you have HPET enabled in the BIOS? You could try disabling it and removing any timecounter settings you may have set in /etc/sysctl.conf letting the operating system choose what it thinks is the best timecounter. HPET doesn't always seem to be very reliable and more so on older hardware.
 
So this solved the clock issue, though backwards from what you were suggesting. I forced it to use HPET and ensured it was enabled in BIOS. Voila, keeps time. However, still seeing the odd refresh behavior.
 
Back
Top