sysctl -a hangs Freebsd terminal

M

mokfarg

Hello, I am trying to diagnose what is causing my Freebsd terminal to hang and the process to be unkillable when executing sysctl -a. I tried troubleshooting by executing sysctl dev , sysctl cpu and those both hang as well. Could anyone recommend how to troubleshoot further to pinpoint what the issue is? Is it a bug, hardware issue, driver etc?

Code: 
truss sysctl -a

write(1,"\n",1)                                  = 1 (0x1)
__sysctl("sysctl.next",6,0x105ef1ae1910,0x105ef1ae1908,0x0,0) = 0 (0x0)
__sysctl("sysctl.name { 1.30.2147482569 }",5,0x105ef1ae1050,0x105ef1ae0bd0,0x0,0) = 0 (0x0)
__sysctl("sysctl.oidfmt kern.ipc.pipe_mindirect",5,0x105ef1ae14c0,0x105ef1ae0bd8,0x0,0) = 0 (0x0)

#execute sysctl -a
#terminal stops responding
#after a few mins I am able to access a terminal session again, and ps -ax shows sysctl -a defunt process
#unable to kill the process
#reboot clears it
#hp diagnostic utility passes
#hp z6 workstation, gpu amd firepro w2100
freebsd-version -kru
14.2-RELEASE-p1
14.2-RELEASE-p1
14.2-RELEASE-p3


sysctl dev
#hangs

dev.pcib.7.%pnpinfo: vendor=0x8086 device=0xa194 subvendor=0x103c subdevice=0x81c6 class=0x060400
dev.pcib.7.%location: slot=28 function=4 dbsf=pci0:0:28:4 handle=\_SB_.PC00.RP05
dev.pcib.7.%driver: pcib
dev.pcib.7.%desc: ACPI PCI-PCI bridge
dev.pcib.6.subbus: 6
dev.pcib.6.secbus: 6
dev.pcib.6.pribus: 2
dev.pcib.6.domain: 0
dev.pcib.6.%domain: 0

sysctl cpu
#hangs
dev.cpu.15.cx_method: C1/mwait/hwc C2/mwait/hwc/bma
dev.cpu.15.cx_usage_counters: 1058653 0
dev.cpu.15.cx_usage: 100.00% 0.00% last 128884us
dev.cpu.15.cx_lowest: C1
dev.cpu.15.cx_supported: C1/1/1 C2/2/41


ps -ax

4322  0- R+    3042:09.26 sysctl dev
9244  1- R       19:17.99 sysctl dev.cpu
9249  1- R       17:12.26 sysctl dev.vgapci
9252  1- R+      16:34.14 sysctl dev.vgapci
9271  2- R<+     14:03.84 su -
9280  3- R+       5:27.73 sysctl dev.cpu
9292  4  Ss       0:00.02 -sh (sh)
9295  4  R        1:02.86 sysctl kern
9296  4  R        0:39.28 sysctl hw
9297  4  R        0:16.93 sysctl net

I am also after unable to shutdown or restart without hard reboot : 
9341 v0  Is+      0:00.02 /bin/sh -o verify /etc/rc.shutdown reboot
 
Does it happen if you grep something?

I did sysctl -a | grep hw.em a lot the past few days no problem, but haven't done sysctl -a ever by itself.
 
Bit of a strange problem... I'm definitely not fully familiar with all of FreeBSD's inner workings, but the only thing which the -a parameter does is list available options within the kernel. So if that fails on your end then this could hint at issues within your kernel (or so I assume!).

Which leads to the obvious question: what kind of system are we talking about? Did you build / (re)configure this yourself or is this a vanilla release?

Next... It's not normal behavior, obviously. And I also can't reproduce anything of the sort either:
Code: 
peter@bsd:/home/peter $ time sysctl -a | wc -l
   11431
    0.09s real     0.00s user     0.00s system
peter@bsd:/home/peter $ su2
root@bsd:/home/peter # time sysctl -a | wc -l
   11688
^ This is actually on CURRENT, which is "unstable by definition", yet as you can see there are no issues at all.

BUT... this did got me thinking... I'm wondering if you're not experiencing hardware related problems OP:
Code: 
peter@bsd:/home/peter $ sysctl kern.disks
kern.disks: da0 cd0
See what I mean? Or what to think about sysctl kern.geom.confxml? If sysctl is trying to retrieve hardware stats yet can't do that for some reason... then you might be having some issues.

So, how to debug this. Best way to do that is to find out where exactly the command is stalling; what's the last thing it retrieves before it stops?
 
Thanks for the replies and help troubleshooting. I rebooted the system and executed sysctl -a again. I executed procstat -kk on the hung process:


Code: 
procstat -kk 3702
  PID    TID COMM                TDNAME              KSTACK                       
 3702 100946 sysctl              -                   pci_find_cap_method+0x15e iommu_get_requester+0x17c device_sysctl_handler+0x1fc sysctl_root_handler_locked+0x91 sysctl_root+0x23e userland_sysctl+0x15e sys___sysctl+0x60 amd64_syscall+0x10e fast_syscall_common+0xf8

Does this provide any clues?
 
mokfarg said:
Thanks for the replies and help troubleshooting. I rebooted the system and executed sysctl -a again. I executed procstat -kk on the hung process:


Code: 
procstat -kk 3702
  PID    TID COMM                TDNAME              KSTACK                      
 3702 100946 sysctl              -                   pci_find_cap_method+0x15e iommu_get_requester+0x17c device_sysctl_handler+0x1fc sysctl_root_handler_locked+0x91 sysctl_root+0x23e userland_sysctl+0x15e sys___sysctl+0x60 amd64_syscall+0x10e fast_syscall_common+0xf8

Does this provide any clues?
Click to expand...
Probably, it reminded me of the following commit in main: https://github.com/freebsd/freebsd-src/commit/6ba2c036a0117ac02f9979b7dc49f15e9c1ea9c9
 
You must log in or register to reply here.
Back
Top