The Case for Rust (in the base system)

cracauer@

cracauer@

Developer

The Case for Rust (in the base system)​


Note that this is about Rust in the base system for userland programs, not for the kernel (yet :)).

Code: 
From: Alan Somers <asomers_at_freebsd.org>
Date: Sat, 20 Jan 2024 16:51:25 UTC


In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system.  To
summarize, the cost is that it would double our build times.  imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain.  That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible.  Here is a list of actual and potential Rust projects
that could benefit from being in-tree.  If anybody else has items to
add, I suggest moving this into the project wiki:

Stuff that could only be written in Rust if it were in base
===========================================================

* ctl-exporter (I started this, but discovered that the CTL stats API is
  unstable, so it can't live in ports.  Instead, I had to do it in C).
  https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34

* fusefs tests.  Absolutely impossible to do in C.  I considered Rust, but went
  with C++ so they could live in base.  They are too closely coupled to
  fusefs(5) to live out-of-tree.
  https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs

* devd.  Currently C++, but imp suggested a rewrite.
  https://github.com/freebsd/freebsd-src/tree/main/sbin/devd

* zfsd.  Currently C++, but I've long pondered a rewrite.  Using Rust would
  make it more testable.
  https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd

* nscd.  Currently C, but confusing and with no test coverage.  I've
  contemplated a rewrite myself, but I don't want to do it in C.
  https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd

* The userland portion of the 802.11ac and Lightning stacks.  scottl suggested
  that these were good candidates for Rust.

* freebsd-kpi-r14-0 .  https://crates.io/crates/freebsd-kpi-r14-0

Stuff that can live in ports, but would be nicer in base
========================================================

* gstat-rs https://crates.io/crates/gstat

* geom-exporter (I've started this, but haven't published it)

* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter

* virtiofsd-rs .  Nobody has yet tried to port it to FreeBSD.  But if the
  connection to bhyve(8) is too intimate, it might be hard to do in ports.
  https://gitlab.com/virtio-fs/virtiofsd

* jail-exporter https://crates.io/crates/jail_exporter

* Various jail managers have been attempted in Rust.  I think these are fine in
  ports, but others like Goran Mekic have opined that they should be moved to
  base instead.

* musikid's pjdfstest rewrite.  I think it would be great to start using this
  to test the base system's file systems.  If the tests themselves lived in
  base, they would be easier to sync with file system development.
  https://github.com/musikid/pjdfstest

* pf-rs.  I suspect that the API isn't very stable.
  https://crates.io/crates/pf-rs

* benchpmc.  The pmc counter names changes between releases.
  https://crates.io/crates/benchpmc

FreeBSD-related applications that are just fine in ports
=========================================================

* fsx-rs.  Unlike pjdfstest, this only tests datapath APIs.  Those are usually
  more stable than control path APIs, so I think there's little to be gained by
  moving this into base. https://crates.io/crates/fsx

* ztop.  It uses ZFS's kstats sysctl interface, which is pretty stable.
  https://crates.io/crates/ztop

* iocage-provision  https://crates.io/crates/iocage-provision

* rsblk https://crates.io/crates/rsblk

* xfuse  https://github.com/KhaledEmaraDev/xfuse

Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
 
devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
Click to expand...

I didn't realize how tiny devd is. Why was this not rewritten in C ages ago?

Edit: Actually, for this entire list... these small things are not appropriate to drag in Rust for at all.

Weirdly, I quite like many parts of Rust (minus the religious people and minus NPM crates.io). I am not entirely opposed to bringing it in compared to any other non-C language (although it does open the floodgates for a lot of mess). However, Rust being crucial for this list is... a bit of a lie surely? Just be honest and repeat after me:

Rust is currently cool. I want to use it to also be cool. I want to pull it into FreeBSD so FreeBSD becomes as cool as me.
Click to expand...

No need to be coy about it ;)
 
kpedersen said:
Edit: Actually, for this entire list... these small things are not appropriate to drag in Rust for at all.
Click to expand...
THIS

building rust fails on many hosts with "low" memory (i.e. <64GB is considered "low memory" by rust devs...) and takes quite long while hogging insane ammounts of ressources. why on earth would someone want to drag in this behemoth to rewrite relatively tiny tools (that aren't even broken)? except of course the old "C is old, therefore we need to replace it with <some new language> that is new and cool" argument...
 
Crivens said:
Building clang is a PITA each time. Double that for what benefit?
And how long untill you need crates.io for building base?
Click to expand...

Why would it be 2x, though? Building LLVM once with C, C++ and Rust frontends would not take 2x.

Maybe right now because different LLVM versions are required (I dunno what exactly has been planned), but long term?
 
sko said:
except of course the old "C is old, therefore we need to replace it with <some new language> that is new and cool" argument...
Click to expand...

I wouldn't say C is broken, but it goes on my nerves. Particularly the lack of ability to have generic algorithms and data structures. At the same time C++ has IMHO gone off the deep end and sized itself (as in mental load size) out of competition with C++20. I didn't even look at C++23 yet. I guess I am a Rust programmer now?
 
cracauer@ said:
At the same time C++ has IMHO gone off the deep end and sized itself (as in mental load size) out of competition with C++20. I didn't even look at C++23 yet. I guess I am a Rust programmer now?
Click to expand...

what[=](do){you}<=>mean?
constexpr std::wstring("Modern") C++(Args...)const std::move(const is)std::span<perfectly>.lock(serviceable&&)!

C++ is pretty much purely in base because it came for "free" with current C compilers. It is sometimes a convenient language but certainly it is messy. Many people tend to overconsume its features and I am not sure why. This culture adds to its messyness. Safety generally got as good as it was going to get around C++11 (C++0x, TR1). Or at least, that's when we no longer had to roll our own shared/weak pointers.

POSIX dictates C99 so I would personally be keen to keep with that for most of FreeBSD.
 
kpedersen Shush! Wash your $EXPLETIVE keyboard out with soap!

Okay, I am not in a position to weight in on that decision, but even if you only need a rust front end, so long as we need two llvm versions this should be a no-go.

This smells of the slippery slope of "it ain't broken, let's fix it".
 
Now would not be the time to introduce Rust in base yet.
Rust ABI changes too often, meaning "moving goal". First, stabilize (finalize) Rust ABI and language spec from some version in the future. My opinion is that it is the earliest time to even discussing about it. Fix the goal first. Then we can start discussing.
 
T-Aoki said:
stabilize (finalize) Rust ABI and language spec from some version in the future.
Click to expand...
Couldn't agree more. I'm reading far too much about these things--along with the difficulty of learning Rust altogether--than it makes me comfortable.
When this has been brought up, I feel it's bandwagon jumping. Here and elsewhere I'm reading more about Zig. Before Zig it was some other language. If we followed all that we'd be flopping around all kinds of different languages today and--talk about instability! This sounds like Linux talk jumping on the latest fad.
 
unitrunker said:
If more hardware support came in the form of rust kernel modules, I'm all for it.
Click to expand...
Unfortunately Rust can't communicate directly with C. It does a much worse job than C++. This means that a Rust kernel module could never plug into the current kernel without someone writing (and subsequently maintaining) a big old fat unsafe binding layer. This is a full time job in its own right.

Linux has been working on their "Rust Abstraction" for a long time and it is still only at the point of supporting a ~190 line PHY network driver (an almost direct clone from the existing working C driver).
 
kpedersen said:
Unfortunately Rust can't communicate directly with C. It does a much worse job than C++. This means that a Rust kernel module could never plug into the current kernel without someone writing (and subsequently maintaining) a big old fat unsafe binding layer. This is a full time job in its own right.

Linux has been working on their "Rust Abstraction" for a long time and it is still only at the point of supporting a ~190 line PHY network driver (an almost direct clone from the existing working C driver).
Click to expand...

so how about node kernel modules? you know, javascript running on a server is considered a 'good idea', so why not running JS in the kernel?

*shudders in disgust*
I'm going to burn my keyboard now...
 
I'd we need something like that, let's take something that is well tested and stable. A simple ABI that won't change in a hurry. Maybe common lisp?
 
mer said:
Rust now, then what, Go?
Click to expand...

If Rust makes it in it is because people are need more than C. With Rust in the base system the same need wouldn't apply again.

Plus Go needs a garbage collector and a big runtime, making it forever unsuitable for the kernel. Rust can be run with a minimal runtime (already developed for embedded Rust) and has no GC.
 
garbage collectors. Yes it's tedious to keep track of your allocations and dealloctions in C/C++, but proper design helps. Languages with GCs tend to create all kinds of allocations that you don't realize and then you need to try and figure out the correct magic words to make the GC run enough but not too much to avoid running out of memory when you allocate faster than you collect.
 
cracauer@ said:
Plus Go needs a garbage collector and a big runtime, making it forever unsuitable for the kernel. Rust can be run with a minimal runtime (already developed for embedded Rust) and has no GC.
Click to expand...
Indeed. That said, I was impressed with TinyGo and that it can run quite well on a GBA.

I am still disappointed by Go's early decision to use a GC. In many ways, Go's preamble stuff is closer to consuming C APIs directly without bindings than most of the competition.

But at least the Go language can do a GC in a portable manner. This was never the case with C (luckily undefined behavior tends to play nice in this regard).
 
  • Like
Reactions: mer
drhowarddrfine said:
Bob Martin of "Clean Code" once remarked that he thought the future belonged to Lisp. Paul Graham was asked if he agreed and he said, "Yes".
Click to expand...

There is no question that Lisp is the only currently existing programming language that you can write truly changeable code in.

But in the base of a Unix system it seems misplaced. Again for GC reasons. And it has no ABI standard. The ABI of the canonical high-performance implementation (SBCL) changes every few weeks. On the bright side, it is really fast to build.

Now, if we want to make a new Lisp machine, I'm all in.
 
cracauer@ said:
If Rust makes it in it is because people are need more than C. With Rust in the base system the same need wouldn't apply again.

Plus Go needs a garbage collector and a big runtime, making it forever unsuitable for the kernel. Rust can be run with a minimal runtime (already developed for embedded Rust) and has no GC.
Click to expand...
And Rust's gc is static whereas Go's gc is dynamic, like that of Java.
 
The only way Rust should be included in FreeBSD, is if it has its own ports tree, which works as a replacement or complement to FreeBSD ports. Also for if there's any future language in the Rust family, to be in that same Rust specific ports tree.

Rust needs to be cleared out of FreeBSD ports, and cloned into its own repository, which can work with or on the side of FreeBSD's default ports/pkgs.

Otherwise, Rust needs to be limited to its own OS. Redox and other operating systems serve that purpose.

As for Crates, it makes more sense that it be library dependencies only, then let the OS or ports tree deal with other ports/packages. Otherwise, Crates becomes overly redundant and tangled.

It makes more sense for Zig or a Zig successor which is not much more intensive than the C family which takes some features from Rust to be in the ports tree. Anything in the C family, including Zig belongs here.
 
You must log in or register to reply here.
Back
Top