Number of CPUs and cores

SirDice

Administrator
Staff member
Administrator
Moderator
I'm looking for one or more sysctl(8) that can get me this information:

Code:
FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads

The number 24 is probably either hw.ncpu (the hardware) or kern.smp.cpus (what the kernel actually uses). But I need to figure out how many physical processors and how many cores each processor for each machine has. I also can't find the sysctl(8) that tells me if Hyper-Threading is turned on (that's the '2 SMT threads').

Surely I could just grep dmesg(8) output but that seems to be unreliable as sometimes the info already got flushed out by other messages.
 
DutchDaemon said:
dmesg.boot won't get flushed ;)
That file doesn't seem to exist on some of the systems I looked at :\

But I think I found a way, kern.smp.active and kern.smp.disabled will tell me if HT is turned on. And I think I can figure out the number of packages by looking at the dev.cpu.*.%location strings (*.SCK0.* and *.SCK1.*). So the number of cores will be hw.ncpu divided by 2 (if HT is turned on) divided by the number of packages I've found.

The rational behind this is that I'm looking for a way to add this info automatically into Zabbix's inventory by crafting a neat UserParameter :e
 
Perhaps kern.sched.topology_spec could give you all the information. Mine says:

Code:
kern.sched.topology_spec: <groups>
 <group level="1" cache-level="0">
  <cpu count="4" mask="f">0, 1, 2, 3</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="4" mask="f">0, 1, 2, 3</cpu>
    <children>
     <group level="3" cache-level="1">
      <cpu count="2" mask="3">0, 1</cpu>
      <flags><flag name="HTT">HTT group</flag><flag name="THREAD">THREAD group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="2" mask="c">2, 3</cpu>
      <flags><flag name="HTT">HTT group</flag><flag name="THREAD">THREAD group</flag></flags>
     </group>
    </children>
   </group>
  </children>
 </group>
</groups>

Code:
[CMD]dmesg | grep FreeBSD/SMP[/CMD]
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads
 
On my new workstation (with AMD cpu) it looks like this: tingo@kg-core1$ dmesg | grep SMP
Code:
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
and tingo@kg-core1$ sysctl dev.cpu | grep location
Code:
dev.cpu.0.%location: handle=\_PR_.P001
dev.cpu.1.%location: handle=\_PR_.P002
dev.cpu.2.%location: handle=\_PR_.P003
dev.cpu.3.%location: handle=\_PR_.P004
and tingo@kg-core1$ sysctl kern.sched.topology_spec
Code:
kern.sched.topology_spec: <groups>
 <group level="1" cache-level="0">
  <cpu count="4" mask="0xf">0, 1, 2, 3</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="4" mask="0xf">0, 1, 2, 3</cpu>
   </group>
  </children>
 </group>
</groups>
Not sure if it helps.
 
I'm looking for one or more sysctl(8) that can get me this information:
FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads [...]
I guess you want to get these informations in a script and not use a C library? IMHO it's better to have sysctls than to compute it from XML provided by kern.sched.topology_spec, because that's error-prone.
Consider to file in a bug report and kindly request to add new read-only sysctl()s:
kern.smp.cpu_packages
dev.cpu_package.*.cores
dev.cpu.*.threads
hw.cpu.packages
hw.cpu.*.cores
hw.cpu.*.threads
Hopefully these are self-explanatory. The hw.* is what the hardware has, the other's are what is in use by the kernel. Maybe it should be hw.acpi.cpu instead?
Keep in mind that the dev.cpu* is dynamic, since it can be changed at runtime on some systems, and in general for jails on any system. I'm not aware of systems with non-uniform CPU packages, but... IIRC there have been very strange combinations in the past by Unisys? AFAIK SMT/HyperThreading on/off is not dynamic, but who knows?
 
I revived this thread because I wanted to share a method that works reliably. The methods discussed in this thread are not reliable. Plus, it's portable. It is a set of executables and a library.

For example, kern.smp.cpu_packages does not exist on my systems; 12.1-RELEASE.
 
Well, answering this question 7 years later is probably a waste of time …

… but I think it’s worth mentioning that, nowadays, SMT does not imply 2 threads per core anymore. I've recently read about new processors that implement 4 threads per core, and even more might be possible in the future.

And second, it also needs to be mentioned that modern CPUs can be more complicated than just “packages / cores / threads”. For example, CPUs can have one or more levels of cache groups, and NUMA architectures are becoming more and more common. Here’s a rather simple example (by today’s standards):
Code:
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 1 package(s) x 2 cache groups x 4 core(s) x 2 hardware threads
That’s not an unusual processor – it’s an AMD Ryzen 2700 that consists of two so-called core complexes (CCX) with localized L3 caches. FreeBSD is NUMA-aware and respects cache groups, which can make a measurable difference in performance.

There are AMD Epyc servers that have two packages, each package is a “Naples” processor with four “Zeppelin” chips (these are AMD codenames), each chip consists of two core complexes, each of theses has four cores, and every core supports two SMT threads. So, such a system has a 2 x 4 x 2 x 4 x 2 topology.

IMHO, the best way to read the CPU topology is using the kern.sched.topology_spec, because that’s why it exists. ;)
All common programming and scripting languages provide functions to parse XML into a native data structure. There are even tools that assist in parsing XML from within shell scripts, for example textproc/xmlstarlet, even though I strongly recommend to use a language like Python for such tasks.

For the above CPU, the XML looks like this:
Code:
<groups>
  <group level="1" cache-level="0">
    <cpu count="16" mask="ffff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15</cpu>
    <children>
      <group level="2" cache-level="3">
        <cpu count="8" mask="ff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7</cpu>
        <children>
          <group level="3" cache-level="2">
            <cpu count="2" mask="3,0,0,0">0, 1</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c,0,0,0">2, 3</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="30,0,0,0">4, 5</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c0,0,0,0">6, 7</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
        </children>
      </group>
      <group level="2" cache-level="3">
        <cpu count="8" mask="ff00,0,0,0">8, 9, 10, 11, 12, 13, 14, 15</cpu>
        <children>
          <group level="3" cache-level="2">
            <cpu count="2" mask="300,0,0,0">8, 9</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c00,0,0,0">10, 11</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="3000,0,0,0">12, 13</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c000,0,0,0">14, 15</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
        </children>
      </group>
    </children>
  </group>
</groups>

If you really want to dissect that with a shell script (not recommended!), a command like this could be used as a starting point:
Code:
$ sysctl -n kern.sched.topology_spec | xml el -v | grep count
groups/group/cpu[@count='16' and @mask='ffff,0,0,0']
groups/group/children/group/cpu[@count='8' and @mask='ff,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='3,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='30,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c0,0,0,0']
groups/group/children/group/cpu[@count='8' and @mask='ff00,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='300,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c00,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='3000,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c000,0,0,0']

Alternatively, awk(1) might be helpful, too:
Code:
$ sysctl -n kern.sched.topology_spec | xml el -v | awk -F/ '/count/ {print int(NF / 2), $NF}' | sort -n | tr -dc '\n 0-9'
1 16  000
2 8  000
2 8  00000
3 2  3000
3 2  30000
3 2  300000
3 2  3000000
3 2  000
3 2  0000
3 2  00000
3 2  000000
Here, the first column gives the level (1 = package, 2 = CCX cache group, 3 = core), and the second column specifies the number of SMT threads on that level. This example (a Ryzen 2700) has 3 levels. As mentioned above, there are systems with 4 or even 5 levels.
 
IIRC the SPARC T-series CPU's had 4 threads/core? And modern ARM have mixed CPUs in a package (A72 & A5x). EDIT And as noted above, IIRC Unisys built mainboards with mixed packages (different CPU classes) in the 90ies?
 
Interesting, thanks.

Any idea how those cores can be identified in user space? For example, say I wanted to bind two application threads to different cpu threads on the same core, or different cores?
 
Any idea how those cores can be identified in user space? For example, say I wanted to bind two application threads to different cpu threads on the same core, or different cores?
You have to figure out the CPU numbers from the CPU topology, then use cpuset(8) and/or rctl(8) to assign the tasks/threads to the respective CPUs. EDIT You can use rctl.conf(5) to permanently apply resource allocation rules.
 
  • Like
Reactions: a6h
You have to figure out the CPU numbers from the CPU topology...

That was kinda the question.
sysctl kern.sched.topology_spec, read it's output & apply the mask entry... E.g.
kern.sched.topology_spec: <groups>
<group level="1" cache-level="3">
<cpu count="4" mask="f,0,0,0">0, 1, 2, 3</cpu>
<children>
<group level="2" cache-level="2">
<cpu count="2" mask="3,0,0,0"> 0, 1</cpu>
<flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
</group>
<group level="2" cache-level="2">
<cpu count="2" mask="c,0,0,0"> 2, 3</cpu>
<flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
</group>
</children>
</group>
</groups>
Thus, cpu0 & cpu1 are sharing the same core via SMT, as well as cpu2 & cpu3.
 
You have to figure out the CPU numbers from the CPU topology...
That can be found in the XML from the kern.sched.topology_spec sysctl. For examle, this line:
Code:
<cpu count="2" mask="c000,0,0,0">14, 15</cpu>
specifies that the threads 14 and 15 are on the same core (and therefore also in the same cache group). Alternatively, depending on programming language, you can also use the mask attribute which is a hexadecimal representation of the bitmap: 0xc000 is binary 1100 0000 0000 0000, i.e. bits 14 and 15 are set. This might be useful when calling certain systemcalls like cpuset_setaffinity(2) directly.
 
Back
Top