What's your fastest buildworld time?

ph0enix · Apr 1, 2022

I just tested 13.0-RELEASE-p10 build on a VM with 64 CPU cores. Here is the result:

Code:

>>> World build completed on Fri Apr  1 15:53:24 EDT 2022
>>> World built in 689 seconds, ncpu: 64, make -j112
--------------------------------------------------------------

real    11m29.304s
user    445m54.395s
sys     27m42.538s

I tested it a few times with higher and lower -j values. There wasn't much difference after around 90.
Normally, this takes about 3 hours on a system with 2 cores and 4GB RAM.

SirDice · Apr 1, 2022

If you have plenty of memory you could set MAKEOBJDIRPREFIX (defaults to /usr/obj) to a ram backed disk. You would need to save it to disk afterwards though, or else the build is gone after a reboot. But if it takes less then 10 minutes to rebuild it might not be worth saving

ph0enix · Apr 1, 2022

That's a good thought! But compiling to a RAM disk didn't make much of a difference in terms of performance:

Code:

real    11m19.541s
user    445m7.060s
sys     26m35.989s

ph0enix · Apr 2, 2022

For comparison (diskinfo -ctv):
da0 (SSD based):

Code:

/dev/da0
    512             # sectorsize
    53687091200     # mediasize in bytes (50G)
    104857600       # mediasize in sectors
    0               # stripesize
    0               # stripeoffset
    6527            # Cylinders according to firmware.
    255             # Heads according to firmware.
    63              # Sectors according to firmware.
    VMware Virtual disk    # Disk descr.
                    # Disk ident.
    mpt0            # Attachment
    No              # TRIM/UNMAP support
    Unknown         # Rotation rate in RPM
    Not_Zoned       # Zone Mode

I/O command overhead:
    time to read 10MB block      0.027765 sec    =    0.001 msec/sector
    time to read 20480 sectors   3.511675 sec    =    0.171 msec/sector
    calculated command overhead            =    0.170 msec/sector

Seek times:
    Full stroke:      250 iter in   0.077214 sec =    0.309 msec
    Half stroke:      250 iter in   0.082961 sec =    0.332 msec
    Quarter stroke:      500 iter in   0.171559 sec =    0.343 msec
    Short forward:      400 iter in   0.134580 sec =    0.336 msec
    Short backward:      400 iter in   0.134877 sec =    0.337 msec
    Seq outer:     2048 iter in   0.249341 sec =    0.122 msec
    Seq inner:     2048 iter in   0.237316 sec =    0.116 msec

Transfer rates:
    outside:       102400 kbytes in   0.366847 sec =   279135 kbytes/sec
    middle:        102400 kbytes in   0.225120 sec =   454869 kbytes/sec
    inside:        102400 kbytes in   0.143715 sec =   712521 kbytes/sec

ramdisk:

Code:

/dev/md5
    512             # sectorsize
    107374182400    # mediasize in bytes (100G)
    209715200       # mediasize in sectors
    0               # stripesize
    0               # stripeoffset
    Yes             # TRIM/UNMAP support
    Unknown         # Rotation rate in RPM

I/O command overhead:
    time to read 10MB block      0.003979 sec    =    0.000 msec/sector
    time to read 20480 sectors   0.243035 sec    =    0.012 msec/sector
    calculated command overhead            =    0.012 msec/sector

Seek times:
    Full stroke:      250 iter in   0.005083 sec =    0.020 msec
    Half stroke:      250 iter in   0.005056 sec =    0.020 msec
    Quarter stroke:      500 iter in   0.008132 sec =    0.016 msec
    Short forward:      400 iter in   0.006814 sec =    0.017 msec
    Short backward:      400 iter in   0.006898 sec =    0.017 msec
    Seq outer:     2048 iter in   0.026359 sec =    0.013 msec
    Seq inner:     2048 iter in   0.026403 sec =    0.013 msec

Transfer rates:
    outside:       102400 kbytes in   0.032122 sec =  3187846 kbytes/sec
    middle:        102400 kbytes in   0.031944 sec =  3205610 kbytes/sec
    inside:        102400 kbytes in   0.032426 sec =  3157960 kbytes/sec

meaw229a · Apr 2, 2022

WOW impressive numbers. The amount of cores on your fingertips and also the 11 min build time.

In comparison:
My Intel I7 9700, 8 core (no further threads) 4.7Ghz, 16 GB Ram with -j8 needs 31 minutes for the world.
My hp t630 Thin Client with 4 core AMD Cpu 2GHz, 8GB Ram with -j4 needs 8 hours 29 minutes.

I'm happy with both times. The I7 gives me time to help myself to a coffee while working and the Thin Client is
in for a night shift.

Cath O'Deray · Apr 3, 2022

I have no record of past durations, here's the most recent:

Code:

% tail -n 6 /usr/src/buildworld.log
      888.73 real       493.85 user       148.95 sys
--- buildworld_epilogue ---
--------------------------------------------------------------
>>> World build completed on Sat Mar 19 02:27:02 GMT 2022
>>> World built in 3398 seconds, ncpu: 4, make -j4
--------------------------------------------------------------
% tail -n 6 /usr/src/buildkernel.log
     1798.32 real      2734.43 user       447.56 sys
--------------------------------------------------------------
>>> Kernel build for GENERIC-NODEBUG completed on Sat Mar 19 02:57:01 GMT 2022
--------------------------------------------------------------
>>> Kernel(s)  GENERIC-NODEBUG built in 1799 seconds, ncpu: 4, make -j4
--------------------------------------------------------------
%

Around fifty-seven minutes. Unusually for me, four jobs.

Why is 888.73 real so much less than World built in 3398 seconds, ncpu: 4, make -j4?

A probe of the computer, the day after the build: <https://bsd-hardware.info/?d=freebsd&probe=dafb1bbb92>

CPU <https://bsd-hardware.info/?d=freebsd&probe=dafb1bbb92#cpu:intel-6-58-9-core-i7-3520m>

Code:

% cat /etc/src-env.conf
WITH_META_MODE=yes
%

I usually run the command below (two jobs) in csh whilst working in Plasma:

 cd /usr/src && time make -j2 buildworld >& buildworld.log && time make -j2 buildkernel >& buildkernel.log && tail buildworld.log && tail buildkernel.log

astyle · Apr 3, 2022

I only did it with a Ryzen 5 1400 about a year ago:

https://forums.freebsd.org/profile-posts/3201/

make -j4 buildworld takes a little over 1.5 hours. I do have 32 GB of RAM on that rig.

meaw229a · Apr 4, 2022

grahamperrin said:
I usually run the command below (two jobs) in csh whilst working in Plasma:

cd /usr/src && time make -j2 buildworld >& buildworld.log && time make -j2 buildkernel >& buildkernel.log && tail buildworld.log && tail buildkernel.log

I really like your approach to this one and was never thinking about it so far. Instead of going with make -j-as-many-jobs-as-possible
go with something like 30% and keep using the computer while building the world.
Next time I will try this for sure.
How responsive is your Plasma while building the world in the background?

Cath O'Deray · Apr 4, 2022

meaw229a said:
… How responsive is your Plasma while building the world in the background?

It's fine, largely thanks to L2ARC.

Some of the time was spent installing FreeBSD to a virtual machine (<https://forums.freebsd.org/posts/562361>), and so on.

grahamperrin said:
… Around fifty-seven minutes. Unusually for me, four jobs. …

With two instead of four, and concurrent use of the desktop environment for much of the time: more than seven hours.

Code:

root@mowa219-gjp4-8570p-freebsd:~ # gh repo sync grahamperrin/freebsd-doc && cd /usr/doc && gh repo sync && git -C /usr/ports pull --ff-only && git -C /usr/src pull --ff-only && cd
✓ Synced the "grahamperrin:main" branch from "freebsd:main"
✓ Synced the "main" branch from grahamperrin/freebsd-doc to local repository
Already up to date.
Already up to date.
root@mowa219-gjp4-8570p-freebsd:~ # echo "" ; date ; uname -sr ; echo "shell: $SHELL" ; echo "" ; git -C /usr/ports pull --ff-only && echo "" ; git -C /usr/src pull --ff-only && set freebsdmaingitcommitcount="`git -C /usr/src rev-list --first-parent --count HEAD`" && echo "freebsd/main (FreeBSD-CURRENT) Git commit count: $freebsdmaingitcommitcount" && set freebsdmaingithash="`git -C /usr/src rev-list --max-count=1 HEAD`" && echo "freebsd/main Git hash: $freebsdmaingithash" && echo "<https://cgit.freebsd.org/src/commit/?id=$freebsdmaingithash>" && echo "– in context: <https://cgit.freebsd.org/src/log/?qt=range&q=$freebsdmaingithash>"

Mon Apr  4 01:12:21 BST 2022
FreeBSD 14.0-CURRENT
shell: /bin/csh

Already up to date.

Already up to date.
freebsd/main (FreeBSD-CURRENT) Git commit count: 254268
freebsd/main Git hash: 50e244964e9b06528b84720e09da7bdf8cec6d32
<https://cgit.freebsd.org/src/commit/?id=50e244964e9b06528b84720e09da7bdf8cec6d32>
– in context: <https://cgit.freebsd.org/src/log/?qt=range&q=50e244964e9b06528b84720e09da7bdf8cec6d32>
root@mowa219-gjp4-8570p-freebsd:~ # cd /usr/src && time make -j2 buildworld >& buildworld.log && time make -j2 buildkernel >& buildkernel.log && tail buildworld.log && tail buildkernel.log
39370.535u 2704.206s 7:18:09.53 160.0%  70628+770k 916821+788927io 188504pf+0w
5508.111u 693.336s 1:18:36.97 131.4%    61501+667k 122940+3775942io 84396pf+0w
--- xattr ---
Building /usr/obj/usr/src/amd64.amd64/tests/sys/fs/fusefs/xattr.full
Building /usr/obj/usr/src/amd64.amd64/tests/sys/fs/fusefs/xattr.debug
Building /usr/obj/usr/src/amd64.amd64/tests/sys/fs/fusefs/xattr
     4450.78 real      5122.84 user       601.17 sys
--- buildworld_epilogue ---
--------------------------------------------------------------
>>> World build completed on Mon Apr  4 08:30:52 BST 2022
>>> World built in 26287 seconds, ncpu: 4, make -j2
--------------------------------------------------------------
ld -m elf_x86_64_fbsd -d -warn-common --build-id=sha1 -T /usr/src/sys/conf/ldscript.kmod.amd64 -r -d  -o radeon_verde_smc_bin.ko radeon_verde_smc_bin.fwo radeon_verde_smc_bin.o
:> export_syms
awk -f /usr/src/sys/conf/kmod_syms.awk radeon_verde_smc_bin.ko  export_syms | xargs -J% objcopy % radeon_verde_smc_bin.ko
objcopy --strip-debug radeon_verde_smc_bin.ko
     4715.89 real      5507.39 user       691.82 sys
--------------------------------------------------------------
>>> Kernel build for GENERIC-NODEBUG completed on Mon Apr  4 09:49:29 BST 2022
--------------------------------------------------------------
>>> Kernel(s)  GENERIC-NODEBUG built in 4717 seconds, ncpu: 4, make -j2
--------------------------------------------------------------
root@mowa219-gjp4-8570p-freebsd:/usr/src # bectl create n"$freebsdmaingitcommitcount"-"$freebsdmaingithash"-a && bectl activate n"$freebsdmaingitcommitcount"-"$freebsdmaingithash"-a && bectl mount n"$freebsdmaingitcommitcount"-"$freebsdmaingithash"-a /tmp/up
Successfully activated boot environment n254268-50e244964e9b06528b84720e09da7bdf8cec6d32-a
Successfully mounted n254268-50e244964e9b06528b84720e09da7bdf8cec6d32-a at /tmp/up
root@mowa219-gjp4-8570p-freebsd:/usr/src # sysrc -R /tmp/up sddm_enable="NO"
sddm_enable: YES -> NO
root@mowa219-gjp4-8570p-freebsd:/usr/src # bectl umount n"$freebsdmaingitcommitcount"-"$freebsdmaingithash"-a && bectl list -c creation
BE                                                 Active Mountpoint Space Created
n250511-5f73b3338ee-d                              -      -          4.94G 2021-11-13 15:43
n252381-75d20a5e386-b                              -      -          6.81G 2022-01-12 23:23
n252450-5efa7281a79-a                              -      -          6.49G 2022-01-14 19:27
n252483-c8f8299a230-b                              -      -          4.84G 2022-01-17 14:24
n252505-cc68614da82-a                              -      -          4.90G 2022-01-18 14:26
n252531-0ce7909cd0b-h                              -      -          5.71G 2022-02-06 12:24
n252997-b6724f7004c-c                              -      -          6.17G 2022-02-11 23:07
n253116-39a36707bd3-e                              -      -          5.66G 2022-02-20 07:03
n253343-9835900cb95-c                              -      -          1.54G 2022-02-27 14:58
n253776-d5ad1713cc3-b                              -      -          3.39G 2022-03-18 09:31
n253861-92e6b4712b5-d                              -      -          3.05G 2022-03-29 03:30
n253861-92e6b4712b5-e                              N      /          1.02M 2022-04-02 16:02
n254268-50e244964e9b06528b84720e09da7bdf8cec6d32-a R      -          175G  2022-04-04 12:27
root@mowa219-gjp4-8570p-freebsd:/usr/src # bectl rename n254268-50e244964e9b06528b84720e09da7bdf8cec6d32-a n254268-50e244964e9-a
root@mowa219-gjp4-8570p-freebsd:/usr/src # bectl list -c creation
BE                    Active Mountpoint Space Created
n250511-5f73b3338ee-d -      -          4.94G 2021-11-13 15:43
n252381-75d20a5e386-b -      -          6.81G 2022-01-12 23:23
n252450-5efa7281a79-a -      -          6.49G 2022-01-14 19:27
n252483-c8f8299a230-b -      -          4.84G 2022-01-17 14:24
n252505-cc68614da82-a -      -          4.90G 2022-01-18 14:26
n252531-0ce7909cd0b-h -      -          5.71G 2022-02-06 12:24
n252997-b6724f7004c-c -      -          6.17G 2022-02-11 23:07
n253116-39a36707bd3-e -      -          5.66G 2022-02-20 07:03
n253343-9835900cb95-c -      -          1.54G 2022-02-27 14:58
n253776-d5ad1713cc3-b -      -          3.39G 2022-03-18 09:31
n253861-92e6b4712b5-d -      -          3.05G 2022-03-29 03:30
n253861-92e6b4712b5-e N      /          1.39M 2022-04-02 16:02
n254268-50e244964e9-a R      -          175G  2022-04-04 12:27
root@mowa219-gjp4-8570p-freebsd:/usr/src # exit
logout
% exit

hruodr · Apr 4, 2022

Nice to have a self build system. Of course, nicer would be to have a self programmed system.

How much time? I do not have much time! I never did buildworld. Sometimes I compile the kernel, no more.

I have some compiled programs in /usr/opt, and I decided to reduce the number to the absolutely necessary by using as much packages as necessary. I recognize that packages are a valuable service.

Do you like to compile, to build? Then become a package/port maintainer! There are a lot of orphaned ports, there are a lot of programs that can become a port.

SirDice · Apr 4, 2022

Make sure you do a make clean, or your build times will be skewed by earlier builds.

Cath O'Deray · Apr 4, 2022

SirDice said:
Make sure you do a make clean,

Will it make the subsequent build slower, or faster?

SirDice · Apr 4, 2022

grahamperrin said:
Will it make the subsequent build slower, or faster?

If you use WITH_META_MODE it will only rebuild what needs to be rebuild. So this will skew your build times. Numbers are only comparable if they're a "full" build, from start to finish. Hence a make clean to clear out any of the previous builds. Or else you're comparing partial build times with complete build times.

MarcoB · Apr 6, 2022

Just got a new system after 18 years, so couldn't resist to post here. A full buildworld finished in 1141 seconds. The kernel was built in exactly 100 seconds. So I'm quite happy

covacat · Apr 6, 2022

if exit code does not count, under one second

astyle · Apr 6, 2022

covacat said:
if exit code does not count, under one second

Did the compilation even start for you?

meaw229a · Apr 7, 2022

MarcoB said:
Just got a new system after 18 years, so couldn't resist to post here. A full buildworld finished in 1141 seconds. The kernel was built in exactly 100 seconds. So I'm quite happy

What's the hardware?

MarcoB · Apr 7, 2022

It's an Intel i9-10900x, 64GB ram, 500GB SSD and a 2TB harddrive.

wb7odyfred · Jan 16, 2024

MarcoB congratulations on your Intel i9 build times! great news to hear.

I do make -j4 buildworld and buildkernel on Raspberry Pi 4B with 8 gigs, with USB 3.0 to NVMe (Ugreen case 1000Mbyte per second) or USB 3.0 to 2.5" SATA SSD (500Mbyte per second)
World takes about 16+ hours; Kernel takes about 5 hours. Yes, slow but gets the job done!

--- kyua.debug ---
objcopy --only-keep-debug kyua.full kyua.debug
--- kyua ---
objcopy --strip-debug --add-gnu-debuglink=kyua.debug kyua.full kyua
4524.20 real 14851.66 user 2066.14 sys
--- buildworld_epilogue ---
--------------------------------------------------------------
>>> World build completed on Tue Jan 16 20:28:29 PST 2024
>>> World built in 57221 seconds, ncpu: 4, make -j4
-------------------------------------------------------------- 57221/3600=15.9 hours

===> usb/g_keyboard (all)
--- all_subdir_usb/g_modem ---
===> usb/g_modem (all)
--- all_subdir_usb/g_mouse ---
===> usb/g_mouse (all)
24.62 real 42.33 user 34.18 sys
--------------------------------------------------------------
>>> Kernel build for GENERIC-VCHIQ completed on Tue Jan 16 20:50:49 PST 2024
--------------------------------------------------------------
>>> Kernel(s) GENERIC-VCHIQ built in 26 seconds, ncpu: 4, make -j4
--------------------------------------------------------------
27.62 real 42.96 user 35.70 sys
Well it was only one file to recompile. So very Fast on Raspberry Pi 4B

time make -j4 buildworld -DNO_CLEAN
time make -j4 buildkernel KERNCONF=GENERIC-VCHIQ -DNO_CLEAN

The important fact is, that FreeBSD allows access to tools and sources to allow a meager system to rebuild world and kernel. So where ever you live in the world, you have access to FreeBSD open source software with BSD license ( ha ha is that what FOSS means?) Instructions to build Kernel
https://ghostbsd-arm64.blogspot.com/2022/09/freebsd-140-compiling-kernel-for.html search word "compile" to find other entries

Argentum · Jan 19, 2024

Just tried to clean build 14.0 on my server with AMD Ryzen 7 3700X 8-Core Processor:

Code:

--- buildworld_epilogue ---

--------------------------------------------------------------
>>> World build completed on Fri Jan 19 20:41:02 EET 2024
>>> World built in 2135 seconds, ncpu: 16, make -j16
--------------------------------------------------------------

________________________________________________________
Executed in   35.58 mins    fish           external
   usr time  511.36 mins  476.00 micros  511.36 mins
   sys time   22.50 mins  180.00 micros   22.50 mins

...and kernel build...

Code:

--------------------------------------------------------------
>>> Kernel build for COBALT completed on Fri Jan 19 20:52:29 EET 2024
--------------------------------------------------------------
>>> Kernel(s)  COBALT built in 157 seconds, ncpu: 16, make -j16
--------------------------------------------------------------

________________________________________________________
Executed in  157.31 secs    fish           external
   usr time   33.07 mins  519.00 micros   33.07 mins
   sys time    2.98 mins    0.00 micros    2.98 mins

This was a rebuild, so some time went for cleaning up.

Cath O'Deray · Jan 19, 2024

Not the fastest, I often make quite heavy use of the computer for things other than building (whilst building world and kernel):

Code:

% grep -C 2 completed\ on /var/log/buildworld.log

--------------------------------------------------------------
>>> World build completed on Sat Jan 13 19:20:22 GMT 2024
>>> World built in 9438 seconds, ncpu: 8, make -j6
--------------------------------------------------------------
% grep -C 2 completed\ on /var/log/buildkernel.log
      878.76 real      4813.64 user       519.78 sys
--------------------------------------------------------------
>>> Kernel build for GENERIC completed on Sat Jan 13 19:35:07 GMT 2024
--------------------------------------------------------------
--------------------------------------------------------------
--
      907.45 real      4957.36 user       518.60 sys
--------------------------------------------------------------
>>> Kernel build for GENERIC-NODEBUG completed on Sat Jan 13 19:50:23 GMT 2024
--------------------------------------------------------------
>>> Kernel(s)  GENERIC GENERIC-NODEBUG built in 1796 seconds, ncpu: 8, make -j16
% grep -C 1 completed\ on /var/log/installkernel.log
--------------------------------------------------------------
>>> Installing kernel GENERIC completed on Sat Jan 13 20:01:21 GMT 2024
--------------------------------------------------------------
--
--------------------------------------------------------------
>>> Installing kernel GENERIC-NODEBUG completed on Sat Jan 13 20:02:13 GMT 2024
--------------------------------------------------------------
% grep -C 2 everything\ completed\ on /var/log/installworld.log
       28.76 real        13.12 user         6.25 sys
--------------------------------------------------------------
>>> Installing everything completed on Sat Jan 13 20:13:01 GMT 2024
--------------------------------------------------------------
      242.95 real       118.35 user        83.64 sys
% date ; uname -aKU
Fri 19 Jan 2024 19:40:20 GMT
FreeBSD mowa219-gjp4-zbook-freebsd 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n267555-e3c7b76fa810-dirty: Sat Jan 13 19:34:36 GMT 2024     grahamperrin@mowa219-gjp4-zbook-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 1500010 1500010
%

HP ZBook 17 G2 <https://bsd-hardware.info/?computer=2ea77aa55d11>, hard disk drive (HTS721010A9E630 1TB), L2ARC on two simple USB flash drives.

winocm · 2024-05-08T01:18:21+0100

Surprisingly, a M3 Max laptop takes around 800 seconds or so when performing a cross world build:

Code:

$ hostinfo
Mach kernel version:
           Darwin Kernel Version 23.4.0: Wed Feb 21 21:44:54 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T6031
Kernel configured for up to 16 processors.
16 processors are physically available.
16 processors are logically available.
Processor type: arm64e (ARM64E)
Processors active: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Primary memory available: 128.00 gigabytes
Default processor set: 775 tasks, 3559 threads, 16 processors
Load average: 3.19, Mach factor: 12.80
$ CPP=/usr/bin/clang MAKEOBJDIRPREFIX=/private/var/tmp/obj ./tools/build/make.py  TARGET=arm64 TARGET_ARCH=aarch64 --host-compiler-type clang --debug -j17 --clean buildworld
Inferred CC as /usr/bin/clang
Running ['/usr/bin/clang', '--version']
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
[...]
--- buildworld ---
--- buildworld_prologue ---
--------------------------------------------------------------
>>> World build started on Tue May  7 17:01:12 PDT 2024
--------------------------------------------------------------
--- _cleanworldtmp ---
[...]
--- buildworld_epilogue ---
--------------------------------------------------------------
>>> World build completed on Tue May  7 17:14:23 PDT 2024
-n >>> World built in 791 seconds, 
ncpu: 16, make -j17
--------------------------------------------------------------
$

What's your fastest buildworld time?

ph0enix

SirDice

Administrator

ph0enix

ph0enix

meaw229a

Cath O'Deray

astyle

meaw229a

Cath O'Deray

hruodr

SirDice

Administrator

Cath O'Deray

SirDice

Administrator

MarcoB

covacat

astyle

meaw229a

MarcoB

wb7odyfred

Argentum

Cath O'Deray

winocm