FreeBSD 15 performance benchmarks

Z

zachw

Michael Larabel at Phoronix recently conducted and published some performance benchmarks of the new 15.0-RELEASE.

1765384974966.png


Highlights include dominant performance from create threads and memory allocation, but also from Zstd compression and x265 video codec.

Curious misses include iPerf and write performance on OpenZFS. I wonder if UFS2 would've done better.

Overall, not bad considering the differential in investment.
 
zachw said:
Highlights include dominant performance from create threads and memory allocation, but also from Zstd compression and x265 video codec.
Click to expand...

glibc's malloc is whack. If Michael would re-run these benchmarks with LD_PRELOAD to tcmalloc Linux would do much better.

Overall pretty disappointing. We used to be pretty equivalent until recently. But as you can see on Phoronix over the last years Linux made several expensive investments that did lead to incremental improvements. Can't believe we are now slower on PyBench.

And all that is on a machine without NUMA.
 
So best performing is 1907.43, FBSD is 1600.25. Delta of 307.18. What does that actually mean? Raw numbers is "about 20%", what is that?

Perf stats are always great to have but I've always had trouble correlating them to real world usage.

I have no doubt that the numbers are accurate for the scenarios, it's the mapping of the scenarios.
 
mer said:
So best performing is 1907.43, FBSD is 1600.25. Delta of 307.18. What does that actually mean? Raw numbers is "about 20%", what is that?

Perf stats are always great to have but I've always had trouble correlating them to real world usage.

I have no doubt that the numbers are accurate for the scenarios, it's the mapping of the scenarios.
Click to expand...

Michael's whole benchmark suite is open source. I found the parts I tried to run reasonably straightforward to use.

He also links to the large tables of raw numbers of his results.
 
  • Like
Reactions: mer
cracauer@ thanks. For me, I've always looked at benchmarks with a grain of salt. I've seen too many that optimize for a cpu/os. I do not think that is what is happening, but I've seen enough in the past to look with interest but not finality.

It always boils down to how does my system feel to me under this load.
 
mer said:
cracauer@ thanks. For me, I've always looked at benchmarks with a grain of salt. I've seen too many that optimize for a cpu/os. I do not think that is what is happening, but I've seen enough in the past to look with interest but not finality.
Click to expand...

Michael strictly does benchmarking with stock OSes and stock machines, no tuning. I'm pretty such that I could improve the Linux results in this run with tcmalloc and setting preemption to none. But I would have to run all that stuff myself.

A networker probably has something to tinker with in the iperf benchmarks with FreeBSD. And as you say, what really matters is how the OS handles things where there is both CPU demand and high I/O at the same time. That is annoying to benchmark.
 
These benchmarks show (only) that without fine tuning, FreeBSD is slower than Ubuntu in some benchmarks, faster in others, and in general a little slower but good enough. But there are so many run-time and compile-time options, and so many different real-world usage scenario, that everything can change in favour of Linux or FreeBSD, if administrators put some effort in fine-tuning the OS and the applications/services. So it can be exciting see a "Formula One" race between OS...
:)


Moreover an OS must be: easy to administer/monitor; secure; reliable; etc.. For example in these benchmarks, Ubuntu is using ext4, while FreeBSD ZFS. The differences in features between the two file systems is so enormous, that raw performances can become not important in certain usage scenario. Or there can be combination of hardware where ZFS is faster.
 
OpenFreeNet said:
These benchmarks show (only) that without fine tuning, FreeBSD is slower than Ubuntu in some benchmarks, faster in others, and in general a little slower but good enough. But there are so many run-time and compile-time options, and so many different real-world usage scenario, that everything can change in favour of Linux or FreeBSD, if administrators put some effort in fine-tuning the OS and the applications/services. So it can be exciting see a "Formula One" race between OS...
Click to expand...

Outside of networking I see more opportunity to tune Ubuntu than FreeBSD for these benchmarks.

Networking is a different matter, we have seen FreeBSD react strongly to some tuning.

There also is the compiler choice which probably is responsible for some of the differences. But you can't compile the Linux kernel with clang (I think) and you can't compile the FreeBSD with gcc anymore (I think). Benchmarking on even terms is hard.
 
cracauer@ said:
Outside of networking I see more opportunity to tune Ubuntu than FreeBSD for these benchmarks.

Networking is a different matter, we have seen FreeBSD react strongly to some tuning.

There also is the compiler choice which probably is responsible for some of the differences. But you can't compile the Linux kernel with clang (I think) and you can't compile the FreeBSD with gcc anymore (I think). Benchmarking on even terms is hard.
Click to expand...
You can compile with clang: https://www.kernel.org/doc/html/latest/kbuild/llvm.html
Some distros do offer kernels compiled with clang, but there isn't much difference there, some wins some losses.
 
I've started some benchmark (with PTS) on my Framework 12 laptop. Arch (BTRFS) and FreeBSD 15.0-RELEASE (ZFS). I'll publish the results in some weeks on my blog (only in french).
For now, i've tried to do same thing manually (every day's task) and there's no visible differences. I'm using Blender alot and it work great on FreeBSD.
Tried Blender with Alpine Linux and FreeBSD go a little faster for OpenGL tasks.
Tested NetBSD (it was my first FOSS OS - nostalgia) and it is very slow.
When the PTS go to end, i'll compile FreeBSD with alderlake CPUTYPE and ports. For now i'm using stock FreeBSD and pkg.
 
being that Ubuntu does spyware type reporting that I was never able to fully disable, I would put up with a 20% mean decrease in performance and stick with BSD as a matter of principle, rather than support, use, or endorse ubuntu You wanna play in the Linux sandbox then choose pure Debian, rather than a commercial bake with a data mining agenda.

Coincidentally, I was looking at the bsd14 v ubuntu (pts) stats less than 48 hours ago, and yes, I agree that there seems to have been a flip in performance between 14 and 15 but not enough to freak out about it.
 
The "Write Test" result was surprising... but they are attributing that result to OpenZFS. This seems to have also showed up in the "Create Files" test as well (?)

Agree -- would have been interesting to see the file system tests against UFS.

"Create Threads" test result rocks though :cool: -- That's awesome!
 
When my Linux install fails after upgrade i'm into a swamp/wasteland.
When my FreeBSD install fails after upgrade it's fixed within minutes.
There a lies damned lies and there are benchmarks.
Mostly i use my PC for firefox & vscode.
Servers are a different thing and need tuning.
 
You must log in or register to reply here.
Back
Top