Running FreeBSD on multiple socket systems

P

Preetpal

By multiple socket systems, I mean systems with motherboards that have more than one CPU installed on the board (like https://www.amd.com/en/products/epyc-7000-series-2-socket-models or https://www.pugetsystems.com/nav/peak/quad_xeon/customize.php). Maybe I am bad at searching but the only thing I have been able to find (besides anecdotes) on running FreeBSD on a multiple socket systems is this video (
) .

I am wondering (purely out of curiosity) if FreeBSD supports multiple socket systems for 64 bit x86 (AMD64) systems? I am also curious to see if anyone if anyone is running FreeBSD on multiple socket systems and if they came across any problems with the setup.
 
sko said:
why shouldn't it?
Click to expand...

Another reason, why I thought it might not work is because I assumed all multiple socket systems have non-uniform memory access. This caused me to believe that it might not work as I had read that a while ago that FreeBSD only recently got NUMA support ().
 
Servers commonly have 2 packages, with multiple cores per package. Almost all the servers I work with on a daily basis have 2 packages. These systems are "quad processor", i.e. 4 packages and are indeed a bit more rare. But they should work nonetheless.

See smp(4) and mptable(1).
 
Indeed, most (or all?) multi-socket machines are NUMA. But you don't need NUMA "support" to run on NUMA machines. Matter-of-fact, tuning applications and workloads for NUMA is very hard (BTDT), and it seems to me that often you are better off not even trying to do NUMA tuning, and let processes run on random cores. I've seen too many times Linux users messing with numactl commands, only to make things run much worse. Or sometimes even completely crash: you can use numactl and ill-behaved applications to crash Linux quite easily.
 
We've been running FreeBSD on multi-socket systems since the original PentiumPro at work. Then on multi-socket P3 systems. Then on multisocket Athlon-MP systems. Then on multi-socket Opteron systems. Then we moved to multi-socket, multi-core Opteron systems. Works great, whether there's 1 CPU, 2, 4, 8, 16, or more. :)

Earliest version I've personally used on multi-socket systems was 4.something, probably 4.4 or thereabouts. (Had 2.2 and 3.1 installed on a laptop at home for testing, but those were single-socket.)
 
I somehow ended up trying to figure out when UNIX or BSD gained multi-processor support...

I had the impression, given that multi-socket/CPU systems were already present in the early days and relatively common not only for "big iron" but even for "small" superminis (e.g. VAX-11/782), UNIX "must" have been able to run on multi-processor systems very early on, but it seems there were various parallel attempts to add SMP support...

According to Wikipedia [1][2], BSD 4.3 was ported in 1986 to the Power 6/32 Tahoe architecture, but the platform suffered from UNIX then lacking proper multi-processor support:
The final issue with the Power 6/32 running Unix was the lack for symmetric multiprocessing: All system calls would have to run on the "Master" processor, forcing a dual processing machine to reschedule a process from the "slave" processor for every system call. The net result of this meant database benchmarks often ran faster on a single processor than a dual.
Click to expand...
With the rise of x86 came BSD386, then came the UNIX wars and BSDs fight for survival in form the 4.4-Lite release, which stalled practically all development for ~3 years - so it is highly unlikely that SMP-support was first introduced to the berkeley-developed BSD in that era, especially because x86 in those days was targeted towards the lowest end computers where SMP wasn't even remotely considered.

Looking at other (proprietary) UNIX/BSD variants, SunOS 4.1.2 from 1991 was the first release with multiprocessor-support [3]. The 4.x branch was based on BSD4.3 - but Sun and AT&T back then diverged from BSD codebase and kept their changes proprietary, so it is unlikely this code ever came back to BSD. Also their UNIX variant was more biased towards System V.

Ultrix OTOH took the best from System V and added it to a BSD codebase. According to Wikipedia, Ultrix-32 could run on various DEC and VAX multiprocessor systems [4]:
The kernel supported symmetric multiprocessing while not being fully multithreaded based upon pre-Ultrix work by Armando Stettner and earlier work by George H. Goble at Purdue University. As such, there was liberal use of locking and some tasks could only be done by a particular CPUs (e.g. the processing of interrupts). This was not uncommon in other SMP implementations of that time (e.g. SunOS).
Click to expand...
Unfortunately I couldn't find any timeline for ultrix releases or when it gained SMP-support, but it must have been somewhere between 1986 and 1991.
Armando Stettner ported OSF/1, which later replaced ultrix at DEC, to the DECstation 3100, which was introduced 1989; so it should be reasonable to further reduce the timeframe to 1986-1989 for SMP support appearing in ultrix.


Regarding FreeBSD, its SMP support didn't came from any of these, as the smp(4) manpages states:
HISTORY
The SMP kernel's early history is not (properly) recorded. It wasdevel-
oped in a separateCVS branch until April 26, 1997, at which pointit was
merged into 3.0-current. By this date 3.0-currenthad already been
merged with Lite2 kernel code.

FreeBSD 5.0 introduced support fora host of new synchronization primi-
tives, anda move towards fine-grained kernel locking rather than
reliance on a Giant kernellock. The SMPng Project reliedheavilyon the
support ofBSDi, who provided reference source code from the fine-grained
SMP implementationfound in BSD/OS.

FreeBSD 5.0 also introduced support for SMP on thesparc64architecture.
Click to expand...

So regarding FreeBSD, the initial SMP support appeared in 1997. But regarding BSD-derived UNIXes in general, history is rather opaque and this information seems to be lost in the void of proprietary software...



[1] https://en.wikipedia.org/wiki/BSD#History
[2] https://en.wikipedia.org/wiki/Computer_Consoles_Inc.#Power_5_and_Power_6_computers
[3] https://en.wikipedia.org/wiki/SunOS#History
[4] https://en.wikipedia.org/wiki/Ultrix#Later_releases_of_Ultrix
 
sko said:
I had the impression, given that multi-socket/CPU systems were already present in the early days and relatively common not only for "big iron" but even for "small" superminis (e.g. VAX-11/782),
Click to expand...

The 11/782 was not a SMP. The two CPUs were very asymmetric: the back CPU had no IO whatsoever. Any process that wanted to do IO had to be moved to the front CPU. Matter-of-fact, I don't know whether the back CPU could run any kernel code.

I think the 11/782 was supported under Ultrix but I don't know how they did it. No idea about running BSD on it; BSD certainly ran well on the 11/780.

I think by the time the 88xx machines and VAX 6xxx machines came out, every serious OS could run multi-processing. It would be stupid to buy an expensive 8840 or 6-CPU 6360 and then throw all but one CPU away. And in those days, you couldn't "partition" or "LPAR" those machines yet, and you couldn't run multiple OSes; that came later, in the mid-90s, from Sun, HP and IBM.
 
You must log in or register to reply here.
Back
Top