syscall-ioctl performance FreeBSD vs Linux

cracauer@ · Jan 7, 2024

Some basic speeds of functions and syscalls for comparison:

Code:

==> amd5950x-freebsd132.log <==
       2.1 nsec/call        2.1 user        0.0 sys: atoi
      51.5 nsec/call       51.5 user        0.0 sys: snprintf
     108.2 nsec/call      108.2 user        0.0 sys: snprintf_float
      19.5 nsec/call       19.5 user        0.0 sys: condvar_signal
      14.9 nsec/call       14.9 user        0.0 sys: mutex_lock_unlock
       6.0 nsec/call        6.0 user        0.0 sys: pthread_mutex_trylock
      22.6 nsec/call       22.6 user        0.0 sys: gettimeofday
      92.8 nsec/call       92.8 user        0.0 sys: strncpy
       9.2 nsec/call        9.2 user        0.0 sys: strchr
     105.8 nsec/call       14.3 user       91.1 sys: getrusage
      54.9 nsec/call       18.3 user       36.6 sys: read
      83.5 nsec/call       16.3 user       67.2 sys: read1bdevzero
     179.5 nsec/call       16.9 user      162.6 sys: read8kdevzero
==> amd5950x-freebsd20240106.log <==
       1.5 nsec/call        1.5 user        0.0 sys: atoi
      63.3 nsec/call       63.3 user        0.0 sys: snprintf
     123.8 nsec/call      123.8 user        0.0 sys: snprintf_float
      17.7 nsec/call       17.7 user        0.0 sys: condvar_signal
      16.8 nsec/call       16.8 user        0.0 sys: mutex_lock_unlock
       5.1 nsec/call        5.1 user        0.0 sys: pthread_mutex_trylock
      24.7 nsec/call       24.7 user        0.0 sys: gettimeofday
     124.5 nsec/call      124.5 user        0.0 sys: strncpy
       5.1 nsec/call        5.1 user        0.0 sys: strchr
     124.0 nsec/call        8.6 user      115.3 sys: getrusage
      68.3 nsec/call       16.5 user       51.6 sys: read
     100.8 nsec/call       22.9 user       77.9 sys: read1bdevzero
     195.9 nsec/call       27.8 user      168.1 sys: read8kdevzero
==> amd5950x-linux5.19.0.log <==
       1.1 nsec/call        1.1 user        0.0 sys: atoi
      33.3 nsec/call       33.3 user        0.0 sys: snprintf
     133.8 nsec/call      133.8 user        0.0 sys: snprintf_float
       8.3 nsec/call        8.3 user        0.0 sys: condvar_signal
       5.8 nsec/call        5.8 user        0.0 sys: mutex_lock_unlock
       4.4 nsec/call        4.4 user        0.0 sys: pthread_mutex_trylock
      17.6 nsec/call       17.6 user        0.0 sys: gettimeofday
      54.3 nsec/call       54.3 user        0.0 sys: strncpy
       4.4 nsec/call        4.4 user        0.0 sys: strchr
     123.9 nsec/call       16.6 user      107.3 sys: getrusage
     102.4 nsec/call       15.3 user       87.0 sys: read
      67.6 nsec/call       13.9 user       53.7 sys: read1bdevzero
     306.7 nsec/call       11.1 user      295.5 sys: read8kdevzero
==> amd5950x-linux6.6.3.log <==
       1.1 nsec/call        1.1 user        0.0 sys: atoi
      33.9 nsec/call       33.9 user        0.0 sys: snprintf
     160.9 nsec/call      160.9 user        0.0 sys: snprintf_float
       8.6 nsec/call        8.6 user        0.0 sys: condvar_signal
       6.0 nsec/call        6.0 user        0.0 sys: mutex_lock_unlock
       4.5 nsec/call        4.5 user        0.0 sys: pthread_mutex_trylock
      18.7 nsec/call       18.7 user        0.0 sys: gettimeofday
      55.6 nsec/call       55.6 user        0.0 sys: strncpy
       4.5 nsec/call        4.5 user        0.0 sys: strchr
     344.1 nsec/call       40.9 user      303.2 sys: getrusage
     317.8 nsec/call       47.5 user      270.2 sys: read
     183.0 nsec/call       24.9 user      158.1 sys: read1bdevzero
     272.2 nsec/call       65.9 user      206.3 sys: read8kdevzero

cracauer@ · Jan 7, 2024

Linux-6.6.3 with all hardware vulnerability mitigations off:

Code:

       1.1 nsec/call        1.1 user        0.0 sys: atoi
      34.0 nsec/call       34.0 user        0.0 sys: snprintf
     145.7 nsec/call      145.6 user        0.0 sys: snprintf_float
       8.5 nsec/call        8.5 user        0.0 sys: condvar_signal
       6.0 nsec/call        6.0 user        0.0 sys: mutex_lock_unlock
       4.5 nsec/call        4.5 user        0.0 sys: pthread_mutex_trylock
      18.7 nsec/call       18.7 user        0.0 sys: gettimeofday
      55.5 nsec/call       55.5 user        0.0 sys: strncpy
       4.5 nsec/call        4.5 user        0.0 sys: strchr
     114.7 nsec/call       13.1 user      101.6 sys: getrusage
      92.5 nsec/call       14.0 user       78.4 sys: read
      69.7 nsec/call       13.0 user       56.7 sys: read1bdevzero
     155.1 nsec/call       13.3 user      141.8 sys: read8kdevzero

Quite the impact, and that is on a rather new AMD CPU, not an old Intel.

bakul · Jan 8, 2024

Looks like newer FreeBSD runs slower while newer Linux runs faster!

cracauer@ · Jan 8, 2024

bakul said:
Looks like newer FreeBSD runs slower while newer Linux runs faster!

Newer Linux is much slower until you turn mitigations off. I honestly didn't expect them to have this much of an impact.

cracauer@ · Jan 8, 2024

cracauer@ said:

Some basic speeds of functions and syscalls for comparison:

Code:

==> amd5950x-freebsd132.log <==
       2.1 nsec/call        2.1 user        0.0 sys: atoi
      51.5 nsec/call       51.5 user        0.0 sys: snprintf
     108.2 nsec/call      108.2 user        0.0 sys: snprintf_float
      19.5 nsec/call       19.5 user        0.0 sys: condvar_signal
      14.9 nsec/call       14.9 user        0.0 sys: mutex_lock_unlock
       6.0 nsec/call        6.0 user        0.0 sys: pthread_mutex_trylock
      22.6 nsec/call       22.6 user        0.0 sys: gettimeofday
      92.8 nsec/call       92.8 user        0.0 sys: strncpy
       9.2 nsec/call        9.2 user        0.0 sys: strchr
     105.8 nsec/call       14.3 user       91.1 sys: getrusage
      54.9 nsec/call       18.3 user       36.6 sys: read
      83.5 nsec/call       16.3 user       67.2 sys: read1bdevzero
     179.5 nsec/call       16.9 user      162.6 sys: read8kdevzero
==> amd5950x-freebsd20240106.log <==
       1.5 nsec/call        1.5 user        0.0 sys: atoi
      63.3 nsec/call       63.3 user        0.0 sys: snprintf
     123.8 nsec/call      123.8 user        0.0 sys: snprintf_float
      17.7 nsec/call       17.7 user        0.0 sys: condvar_signal
      16.8 nsec/call       16.8 user        0.0 sys: mutex_lock_unlock
       5.1 nsec/call        5.1 user        0.0 sys: pthread_mutex_trylock
      24.7 nsec/call       24.7 user        0.0 sys: gettimeofday
     124.5 nsec/call      124.5 user        0.0 sys: strncpy
       5.1 nsec/call        5.1 user        0.0 sys: strchr
     124.0 nsec/call        8.6 user      115.3 sys: getrusage
      68.3 nsec/call       16.5 user       51.6 sys: read
     100.8 nsec/call       22.9 user       77.9 sys: read1bdevzero
     195.9 nsec/call       27.8 user      168.1 sys: read8kdevzero
==> amd5950x-linux5.19.0.log <==
       1.1 nsec/call        1.1 user        0.0 sys: atoi
      33.3 nsec/call       33.3 user        0.0 sys: snprintf
     133.8 nsec/call      133.8 user        0.0 sys: snprintf_float
       8.3 nsec/call        8.3 user        0.0 sys: condvar_signal
       5.8 nsec/call        5.8 user        0.0 sys: mutex_lock_unlock
       4.4 nsec/call        4.4 user        0.0 sys: pthread_mutex_trylock
      17.6 nsec/call       17.6 user        0.0 sys: gettimeofday
      54.3 nsec/call       54.3 user        0.0 sys: strncpy
       4.4 nsec/call        4.4 user        0.0 sys: strchr
     123.9 nsec/call       16.6 user      107.3 sys: getrusage
     102.4 nsec/call       15.3 user       87.0 sys: read
      67.6 nsec/call       13.9 user       53.7 sys: read1bdevzero
     306.7 nsec/call       11.1 user      295.5 sys: read8kdevzero
==> amd5950x-linux6.6.3.log <==
       1.1 nsec/call        1.1 user        0.0 sys: atoi
      33.9 nsec/call       33.9 user        0.0 sys: snprintf
     160.9 nsec/call      160.9 user        0.0 sys: snprintf_float
       8.6 nsec/call        8.6 user        0.0 sys: condvar_signal
       6.0 nsec/call        6.0 user        0.0 sys: mutex_lock_unlock
       4.5 nsec/call        4.5 user        0.0 sys: pthread_mutex_trylock
      18.7 nsec/call       18.7 user        0.0 sys: gettimeofday
      55.6 nsec/call       55.6 user        0.0 sys: strncpy
       4.5 nsec/call        4.5 user        0.0 sys: strchr
     344.1 nsec/call       40.9 user      303.2 sys: getrusage
     317.8 nsec/call       47.5 user      270.2 sys: read
     183.0 nsec/call       24.9 user      158.1 sys: read1bdevzero
     272.2 nsec/call       65.9 user      206.3 sys: read8kdevzero

For completeness, here is 14.0p4:

Code:

*********************************
*******   ioctl test      *******
*********************************
Total time = 172702488.000000 us
2:52.70 172.70 real 26.76 user 145.93 sys 99% CPU 0/89 faults

       1.6 nsec/call        1.6 user        0.0 sys: atoi
      51.8 nsec/call       51.8 user        0.0 sys: snprintf
     113.6 nsec/call      113.6 user        0.0 sys: snprintf_float
      15.2 nsec/call       15.2 user        0.0 sys: condvar_signal
      11.4 nsec/call       11.4 user        0.0 sys: mutex_lock_unlock
       4.5 nsec/call        4.5 user        0.0 sys: pthread_mutex_trylock
      22.9 nsec/call       22.9 user        0.0 sys: gettimeofday
      94.3 nsec/call       94.3 user        0.0 sys: strncpy
       9.8 nsec/call        9.8 user        0.0 sys: strchr
     112.6 nsec/call       17.7 user       94.8 sys: getrusage
      69.5 nsec/call       23.7 user       45.6 sys: read
      81.7 nsec/call       13.7 user       68.0 sys: read1bdevzero
     148.0 nsec/call       17.6 user      130.3 sys: read8kdevzero