Optimisations for eg make / ports building / llama

Alain De Vos

Alain De Vos

Here is some tuning i did on Redcore linux,
its rather self explaining.
Purpose I want to do same on FreeBSD.
Please advice actions to take

redcorelinux /etc/portage/make.conf # cat 09custom
Code: 
LC_MESSAGES="C.UTF-8"
VIDEO_CARDS="nvidia"
MAKEOPTS="-j10 -l10"
NINJAOPTS="-j10"
CFLAGS="${CFLAGS}     -mtune=native"
CXXFLAGS="${CXXFLAGS} -mtune=native"

redcorelinux /etc/portage/make.conf # cat 00-cpuflags
Code: 
# This file has been automatically generated, do not edit.
CPU_FLAGS_X86="mmx mmxext sse sse2 sse3"

redcorelinux /etc/portage/package.use # cat 00cpu-flags
Code: 
*/* CPU_FLAGS_X86: aes avx avx2 avx_vnni bmi1 bmi2 f16c fma3 mmx
mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3 vpclmulqdq

Build llama-cpp
Code: 
[ebuild  N     ] sci-misc/llama-cpp-0.7.0::guru 
USE="cuda openmp openssl -blas -blis -examples -flexiblas -openblas -opencl -rocm -test -vulkan"
AMDGPU_TARGETS="gfx90a gfx908 gfx942 gfx1030 gfx1100 gfx1101 gfx1200 gfx1201 
-gfx803 -gfx900 -gfx906 -gfx940 -gfx941 -gfx1010 -gfx1011 -gfx1012 -gfx1031 -gfx1102 -gfx1103 -gfx1150 -gfx1151"
CPU_FLAGS_X86="avx avx2 avx_vnni bmi2 f16c fma3 sse4_2 -amx_bf16 -amx_int8 -amx_tile
 -avx512_bf16 -avx512_vnni -avx512f -avx512vbmi"
 
It is way less "self explicatory" then you might think. You list all kinds of CPU instruction set support options but then setup some list of AMDGPU_TARGETS, i.e. you want to infer on the GPU, in which case your CPU optimization is irrelevant as the GPU does all the heavy lifting. Also, FreeBSD does not support cuda, so you either have to
  • install a linux chroot/jail/vm
  • use Vulkan, which doesn't distinguish target GPUs but has the video driver optimize the code at load-time
  • use CPU inference

In case you want to use a linux implementation, we have a rocky linux skeleton in the ports collection to do a chroot setup (handbook entry), while jail and vm deployments allow you to use whatever linux distribution you feel confortable with.

In case you want to use the CPU exclusively, I wouldn't bother with the fine tuning. llama-cpp implements all runtime critical code inside a DLL, and it's build logic has the ability to build this DLL redundantly for all upstream supported CPUs, perform an instruction set support detection at application startup, and then load the most appropriate dll for the system it's being run on. This ability just isn't enabled in the port.. or rather, it's disabled purposefully, most likely to workaround the build to crash with an internal compiler error on llvm22.

You can cherry pick / merge this commit on a ports' 2026Q4 checkout to enable that feature on the ggml / llama-cpp ports and workaround the llvm bug by using llvm21 rather then llvm22, and then rebuild both ggml and llama-cpp. It won't give you much wonders though, as the avx512(bf16/vnni) paths are mostly not implemented yet in llama-cpp and therefore optimizing for modern CPUs barely gives an advantage over targetting decade old CPU series, or at least on my xeon 6530 the difference between the sapphire-rapids code path and the haswell code path is like 5% performance only.
 
make.conf has a man page . Also, even in linux, LC_MESSAGES is an environment variable that is dereferenced to print diagnostics at application run time. It should be either in your login.conf, your shrc/zshrc/bashrc/... or in your .profile, rather then in make.conf
 
You must log in or register to reply here.
Back
Top