The Case for Rust (in the base system)

patmaddox said:
I'm in the same boat, and I don't really get it - LLVM is fully backwards-compatible, right? So it should be possible to only build the highest version LLVM needed, not need to build three different versions.
Click to expand...

Not if you use it as a library collection. The library APIs change heavily with each version.
 
Re: last month's session on integration with Rust (not scheduled, I don't know which of the two days),

"… While integrating Rust into FreeBSD presents challenges, particularly due to its rapid release cycle, the potential benefits in terms of security and modernization make it a worthy endeavor. The FreeBSD community is encouraged to actively engage with the Rust ecosystem to address these challenges and leverage Rust’s capabilities to enhance the FreeBSD operating system."
 
Does it mean rewriting (porting) base to Rust begins?
Or aggressively accelerate developments using Rust on ports, not yet for base?
Anyway, massive discussions with LLVM persons would be needed to incorporate Rust frontend into LLVM.
 
More generally (summit highlights, the first of four bullet points):

"Innovation and Modernization: Integrating modern tools and technologies like Rust and enhancing container support were viewed as critical …"

T-Aoki said:
Does it mean rewriting (porting) base to Rust begins?
Click to expand...

Support for Rust in kernel and userland are wanted but not critical, according to the planning document for 15.0.
 
T-Aoki said:
Does it mean rewriting (porting) base to Rust begins?
Or aggressively accelerate developments using Rust on ports, not yet for base?
Anyway, massive discussions with LLVM persons would be needed to incorporate Rust frontend into LLVM.
Click to expand...
In lang/rust , there's already options to build it with bundled version of LLVM or the host's devel/llvm port.

There's already a few ports that are in fact heavy on Rust, and that annoys me to no end.
 
astyle said:
In lang/rust , there's already options to build it with bundled version of LLVM or the host's devel/llvm port.
Click to expand...
If I recall correctly, the option is often broken and sometimes dropped.
Does it work fine nowadays? And can it track default LLVM in time?
And incorporating Rust frontend into LLVM itself would help defaulting per-crate shared object (*.so) and C ABI for FreeBSD.
 
T-Aoki said:
If I recall correctly, the option is often broken and sometimes dropped.
Does it work fine nowadays? And can it track default LLVM in time?
And incorporating Rust frontend into LLVM itself would help defaulting per-crate shared object (*.so) and C ABI for FreeBSD.
Click to expand...
Well, that 'bundled llvm vs the ports one' creates circular dependencies, which is why I turn it off so that the rust port uses the bundled llvm. Yeah, that does make for nearly double the compile time for the rust port, but avoids circular dependencies.

Unfortunately, with Rust, there's a LOT of code/crate replication, so it gets pretty bloated for no good reason. It's the second coming of Java.

I'd vote for actually removing Rust from FreeBSD, but it's a bit too late in the game. ?
 
astyle said:
Unfortunately, with Rust, there's a LOT of code/crate replication, so it gets pretty bloated for no good reason. It's the second coming of Java.
Click to expand...
Three versions of Java compile faster for me than a single LLVM or Rust ;) I gave up on Vscodium due to the Electron shenanigans, and won't even attempt any version of Chrom(e/ium).
 
Pretty ill-informed article:

“So a Rust crate is a collection of code, or it is a blob of precompiled code. Yep. Wait, what? A blob of precompiled code? As in, a type of blackbox?” no. A binary target is executable. A library target is linkable. Overwhelming majority of crates are open source.

The rest of the criticisms apply equally to FreeBSD ports tree.

So it boils down to an apparent misunderstanding, and pointing out the well-known challenges of large scale open source communities.

The counter-article “My Concern with FreeBSD, or a case for Rust” goes “Rust avoids memory errors. FreeBSD uses C and has had lots of memory errors leading to vulnerabilities. The ports tree is contributed by randos who aren’t even the third-party developers, how could it ever be secure. Plus some of those ports use crates!”
 
patmaddox said:
Pretty ill-informed article:

“So a Rust crate is a collection of code, or it is a blob of precompiled code. Yep. Wait, what? A blob of precompiled code? As in, a type of blackbox?” no. A binary target is executable. A library target is linkable. Overwhelming majority of crates are open source.

The rest of the criticisms apply equally to FreeBSD ports tree.

So it boils down to an apparent misunderstanding, and pointing out the well-known challenges of large scale open source communities.

The counter-article “My Concern with FreeBSD, or a case for Rust” goes “Rust avoids memory errors. FreeBSD uses C and has had lots of memory errors leading to vulnerabilities. The ports tree is contributed by randos who aren’t even the third-party developers, how could it ever be secure. Plus some of those ports use crates!”
Click to expand...
Possibly I'm mis-understanding, but my current guess is that crates are similar with individual port(s) which any of its(their) commits are done by submitter (author) == maintainer == committer case.

For ports, an example as the longest way.

If I, neither a maintainer nor committer, submit a patch to a port on bugzilla.freebsd.org, it should be triaged (sometimes automatic as of MAINTAINER data in its Makefile, sometimes manually by triagers).

Then, the patch must be approved by the maintainer or wait for maintainer timeout longer than (basically) 2 weeks. In the mean time, anybody in the world having connection with Internet can check the patch and, if in doubt, tell the maintainer not to approve it (to do this, need registered account) or attach proper patch to be audited.

If the maintainer doesn't have a commit bit to ports tree, a committer is required to be assigned by triager or any of committer who found the PR.

The committer assigned has responsibility to audit and test the patch before committing.

And to be a committer, the person must be approved by core team.

Is there any rule like this for crates?
If not, I cannot consider crates safe by itself. Each individual crate used by any ports should have a responsible maintainer on ports and committed as a port per a crate.
 
T-Aoki said:
Is there any rule like this for crates?
Click to expand...

In your developer / submitter / committer scenario, crates.io (and vast majority of open source package repos) have only the first two layers.

If you think that ports committers thoroughly audit the software source before accepting a patch, I am afraid you are mistaken. They review the ports patch itself - the build instructions - but as to the source code it pulls… nope. There are plenty of ports where the maintainer / committer doesn’t even check if the software runs after a successful build.

sko said:
I thought the discussion here is about rust *in base* - and pulling precompiled binaries from random sources into base should be an absolute no-go
Click to expand...

1. I was responding to an article that someone shared.
2. Nobody is advocating for pulling precompiled binaries from random sources. That’s not how crates work. Any third-party Rust programs in base would have their code in contrib/ same as any other software.

That second point brings up a pretty big logistical issue: you need to have all of the source in contrib; you need all of the dependency source in contrib; and you need to verify that all of the licenses are acceptable. And the way that Rust (and Python, and Go, and Ruby, and any other 21st century programming language with a package manager) works is that every program has its own dependency environment, listing the dependencies it uses and their versions. Two programs may have completely incompatible versions of a single dependency, so what do you do then? Either patch one to make it work with a newer one and hope it gets upstreamed, or run both dependencies alongside one another... it gets messy. Effectively each of these programs statically links to its own versions of dependencies.

So I think it's unlikely to see a lot of Rust software in contrib/. If Rust gets used in base, it'll be mostly by FreeBSD developers who pull up src/, (re-)write something, and commit it. Which I think that's pretty interesting in terms of providing a reliable programming environment for Rust. Any pulled in dependencies should get thoroughly audited, and most of all we would end up agreeing on versions of dependencies that can be used across the board. So it becomes a platform for building Rust programs that, while a bit limited and not on the cutting edge, is predictable and reliable. To an extent, Ports tree has addressed some of this - look at how Python dependencies are handled. Each dependency has its own port, so the versions are consistent. This requires coordination, and sometimes there are multiple major or minor versions, but all the versions are pinned.

Another possibility though is that developers spend a bunch of time bickering about what dependencies are allowed, what versions to use (even of rustc) etc. End users get confused and frustrated why they don't have access to particular features even though it ships with Rust. I wasn't around for the time when Perl was in base, but it brought about some of the same challenges. So the closest current example we have right now, I think, is lua. FreeBSD ships with a version of lua (/usr/libexec/flua) and I think it may even have some libraries. But it's not really intended for end users, and I'm not sure there's a way to install Lua dependencies with it. So there ends up being a ports version that is intended for end users.
 
patmaddox said:
And the way that Rust (and Python, and Go, and Ruby, and any other 21st century programming language with a package manager) works
Click to expand...

We have had dependency-first aggregation languages way before the 21st century. Javascript and Python are good examples.

Large projects (such as operating systems) are not written in these languages for a variety of reasons, particularly relating to the quick'n'easy^W careless way they handle external dependencies (and at absured volume).
 
patmaddox said:
They review the ports patch itself - the build instructions - but as to the source code it pulls… nope.
Click to expand...
It's my assumption/expectation about patches for build fix and/or updates. The full audit should better done for the first import, but it would strongly depend on the complexity and/or scale (in code base, upstream team).


patmaddox said:
There are plenty of ports where the maintainer / committer doesn’t even check if the software runs after a successful build.
Click to expand...
I know, unfortunately. But if the assigned committer actually using the port, the committer would surely test-run at least fot his/her configuration.

patmaddox said:
That second point brings up a pretty big logistical issue: you need to have all of the source in contrib; you need all of the dependency source in contrib; and you need to verify that all of the licenses are acceptable.
Click to expand...
As currently works for switching GPL'ed component to alternatives in BSD-compatible license are ongoing and coming near to be finished, it would be quite important.

patmaddox said:
So I think it's unlikely to see a lot of Rust software in contrib/. If Rust gets used in base, it'll be mostly by FreeBSD developers who pull up src/, (re-)write something, and commit it. Which I think that's pretty interesting in terms of providing a reliable programming environment for Rust. Any pulled in dependencies should get thoroughly audited, and most of all we would end up agreeing on versions of dependencies that can be used across the board.
Click to expand...
It is a part of what I expect when Rust is incorporated into base.
The import should be to allow step-by-step (re-)implementations.
I think it should clearly and strongly be object to Rust philosophy, but legacy Unix one. Obeying Rust philosophy would force us reimprementing all at once except quite limited part that Rust cannot support (like still-remaining asm codes), thus, impossible for limited community like FreeBSD.
What I fear is that FreeBSD becomes huge single binary including kernel and userland (excluding loader) to get maximum benefit from security model of Rust. Yes, it would be most secure implementation as Rust guys love.

patmaddox said:
So it becomes a platform for building Rust programs that, while a bit limited and not on the cutting edge, is predictable and reliable.
Click to expand...
I think it should be the way to go if Rust in base is actually proceeded.
But I "feel" persons who want Rust in base are objecting to it.
Just my feeling, but they seems to be requesting what are currently done in ports into base.

patmaddox said:
I wasn't around for the time when Perl was in base, but it brought about some of the same challenges.
Click to expand...
If I recall correctly, perl was used for tests and code morphing (convert, merge, cut-off, etc). So perl was omitted from base and developers responsible to codes (in C, m4, yacc, lexx, sh and tables basically) install perl from ports and prepare already morphed code to commit.

patmaddox said:
FreeBSD ships with a version of lua (/usr/libexec/flua) and I think it may even have some libraries.
Click to expand...
As far as I know, it is used for loader only. Don't know about future plans.
 
I wonder whether library-ism is so bad. After all if you do all the work yourself in C without extended libraries you could do the same in Rust, no?
 
cracauer@ said:
I wonder whether library-ism is so bad. After all if you do all the work yourself in C without extended libraries you could do the same in Rust, no?
Click to expand...
I don't think "library-ism" is bad.
But standard C libraries are in base and managed (at least, moderated) by FreeBSD developers.
For Rust-in-base case, all crates used in base should be managed as such and built as (basically) one shared object per crate and linked/called as a library, just as C libraries.
This would be significant difference between in-base and in-ports cases.
 
cracauer@ said:
I wonder whether library-ism is so bad. After all if you do all the work yourself in C without extended libraries you could do the same in Rust, no?
Click to expand...
It is almost certainly a culture thing. A decent developer will be responsible with dependencies.

Non-C languages will generally always need more dependencies for the bindings where i.e SWIG/bindgen is ineffective. Much of crates.io is bindings.

But if you look around at almost any Rust project, it pulls in a lot of needless shite. Look at gstat-rs (from your first post) for example.


It is a little ridiculous.
 
You must log in or register to reply here.
Back
Top