Ars Technica article focused on Wireguard regarding FreeBSD

[mark_j]

No luck involved. Another Freebsd committer reached out to Donenfeld (lead author of Wireguard) with concerns about the implementation.
The two of them plus an Openbsd developer that worked on the Openbsd WG implementation then worked together to come up with an alternate implementation.

Nope not as I understand it.
See Kyle Evans post: https://docs.freebsd.org/cgi/getmsg.cgi?fetch=0+0+archive/2021/freebsd-arch/20210321.freebsd-arch

To quote Donenfield to Kyle Evans after he pulled the entire code out of the tree: "I think what you describe is a great plan. I think everybody realizes
at this point that the original code base from the original author never should have been merged."

So the process is flawed, it was lucky that Evans found these bugs. The fact Evans thinks now it is best outside the main, is evidence that this code should never have got to where it did, a release candidate. Quality control of security-scale software needs more than what FreeBSD just showed for Wireguard.

 

[Mjölnir]

FWIW I guess we'll have this covered in it's own Office Hours. From yesterday's <freebsd-hackers>:

[...] This incident highlights the need to do this work more
proactively, and to maintain a robust, multi-layered development process
that can catch problems as they fall through the cracks.

Over the next month the FreeBSD Core Team will lead a discussion on
appropriate pre-commit testing, static analysis, code review, and
integration policies to avoid a repeat of this situation and to continue
improving FreeBSD's code quality. We know there will be challenges in
key areas, such as third-party device drivers, and components of the
system where fewer developers have sufficient expertise. The FreeBSD
Foundation has full-time staff members participating in significant code
review today, and is committed to supporting the needs identified by the
Core team and the developer community for this effort.

We look forward to input from the community on our proposals for updated
policies as we move forward, maintaining high code quality as a core
value for FreeBSD.

I'm not sure if I should seriously suggest to add an interview done by a team of professional, highly skilled psychologists to check the so-called soft skills, before granting a commit bit to a developer.

 

[vigole]

I'm not sure if I should seriously suggest to add an interview done by a team of professional, highly skilled psychologists to check the so-called soft skills, before granting a commit bit to a developer.

That would be great! But I'm more in favour of a department of inquisition.

I won't draw a conclusion, but I wonder which departments made ASA to issue this statement:

ASA: The ASA Statement on p-Values: Context, Process, and Purpose
APA: P-values under question
more:
Nature: Scientists rise up against statistical significance
Nature: 1,500 scientists lift the lid on reproducibility

 

[Snurg]

Quality control of security-scale software needs more than what FreeBSD just showed for Wireguard.

What kind of security clearance would a guy get who is in debt over both his ears, has proven that he has zero loyalty, zero sense of responsibility, would even (literally) sell his mother, is willing to do heinous crimes to satisfy his greed?

How responsible is it to push such code into a release candidate, not caring a s**t about the consequences he causes for his clients Netgate and FreeBSD, just for getting his payment early, due to his apparently pressing financial situation?

Wouldn't such a guy be the dream for organized crime or (foreign) services?
Smuggling in an innocuous bug with hidden backdoor for a relatively small bakshish, shouldn't be this an easy thing for a money-starved greed-motivated person without any sense of responsibility?

Would such a guy get clearance for a job in a security-aware organization for their most-security-relevant software?

 

[Mjölnir]

That would be great! But I'm more in favour of a department of inquisition.

Blaming s/o who's code is cluttered with #ifdef's and constraints unconditionally return true has nothing to do with inquisition. As noted above a few times, critical questions on Phabricator were simply ignored, as well as the offer to help by e-mail. IIRC the average LoC a developer writes is 8/day (!!!). 40 kLoC were commited in just a few month, meeting every notion of C's worst practices. All that together is just another proof of an obnoxious, yukky character. Period.

I won't draw a conclusion, but I wonder which departments made ASA to issue this statement:

ASA: The ASA Statement on p-Values: Context, Process, and Purpose
APA: P-values under question
more:
Nature: Scientists rise up against statistical significance
Nature: 1,500 scientists lift the lid on reproducibility

I welcome your critical mind, but I can't relate any of this to the issue. Please elaborate.

 

[richardtoohey2]

Not the same thing at all, but malicious activity affecting PHP and their processes: https://www.theregister.com/2021/03/29/php_repository_infected/

 

[Jose]

Nope not as I understand it.

...
Fortunately, two weeks before FreeBSD 13.0 was due to be released,
FreeBSD core developer Kyle Evans emailed the list about integrating
wireguard-tools (wg(8) and such). In the ensuing discussion I mentioned
that we really need to get the actual if_wg kernel implementation up to
snuff. We took the conversation to IRC, and agreed that we should work
on figuring out what to do before the release date...

[ANNOUNCE] WireGuard for FreeBSD in development for 13.y – and a note of how we got here

 

[mark_j]

FWIW I guess we'll have this covered in it's own Office Hours. From yesterday's <freebsd-hackers>:

[...] This incident highlights the need to do this work more
proactively, and to maintain a robust, multi-layered development process
that can catch problems as they fall through the cracks.

[...]

And with regard to this issue, this is exactly what we should see; the core developers reacting to an obvious failure of process.

This is not an individual developer's issue, this is about processes and procedures.
I know open source walks a fine line between keeping people interested ("we just want to code") and staving off software failures but better processes are obviously required.
I do sympathise with Kyle Evans; he's caught in the middle of a fiasco.

 

[Jose]

I'm not sure if I should seriously suggest to add an interview done by a team of professional, highly skilled psychologists to check the so-called soft skills, before granting a commit bit to a developer.

Yeah, and then immediately disqualify anyone the "highly skilled" psychologists approve.

 

[mark_j]

[ANNOUNCE] WireGuard for FreeBSD in development for 13.y – and a note of how we got here

Sigh, you miss the point. It's in RC, not some branch somewhere being developed, it's in RC.
"In the ensuing discussion I mentioned that we really need to get the actual if_wg kernel implementation up to snuff". So, someone waited for someone else to trigger what could have eventually gone to release? Ok, if that's not luck then it is horrendous software control.

 

[Jose]

I do sympathise with Kyle Evans; he's caught in the middle of a fiasco.

Yeah, me too. I understand why Donenfeld had to make a big deal out of this, but I can see how Evans got stuck in the middle of a bad situation. That is a bummer.

 

[vigole]

I welcome your critical mind, but I can't relate any of this to the issue. Please elaborate.

My point is that psychologists are not in a position to make a decision who should be on a team or not. It's up to other team members, e.g. Core and Committers. Of course based on whatever criteria they're comfortable with. It's up to them. Team decides whether Terry Davis belongs to the team or not. Especially in volunteer based open source projects. At least scientists and engineers gave us Maxwell's equation, and UNIX kernel. What are psychology achievements? -- beside calling people lunatics, and lock them up in booby houses.

 

[Morris]

I have to say that I see some quite unhelpful ad hominem attacks on the messenger here.

Ars Technica’s Jim Salter is not just some writer that wants to score some points here. You could argue he is not even a writer. Principally he is an IT guy, a sysadmin and consultant working in the storage space. He just happens to have a good pen too, unfortunately rare in the IT sector.

That good pen has made him able to write very compelling articles about ZFS and OpenZFS in particular that will have made thousands consider FreeBSD as their operating system.

• Will ZFS and non-ECC RAM kill your data?
• ZFS tuning cheat sheet
• A detailed look at Ubuntu’s new experimental ZFS installer
• ZFS 101—Understanding ZFS storage and performance
• OpenZFS 2.0 release unifies Linux, BSD and adds tons of new features

By making complex elements of ZFS understandable to a wider audience he has done more for FreeBSD than most of us on this forum and definitely more than I ever did (or will be able to). When I read this article I read it as written by someone who cares about FreeBSD because he has to work with it on a daily basis. His favourite open source NAS, XigmaNAS, is developed on top of FreeBSD.

I think he was just as surprised as I was to see that code that is demonstrably poor made it so far in the release process for FreeBSD 13 without any alarm bells. Many of the issues should have been filtered out in the review stage.

He could have done what many other journalists have done and just follow the narrative from one side and join in the mud slinging. Instead he chose to have the actual code checked to see if the claims were true. He spoke to many players, not only NetGate, or only Jason Donenfeld, or only Kyle Evans, or only Matt Macy. He tried to get a clear picture of what really happened.

Now, you can do two things. You can either become defensive and pretend the code wasn’t sloppy at all. Or claim that a committer shutting down parts of the review process himself is not a problem. Or deny that demonstrably bad code making it all the way into a second Release Candidate of an actual major RELEASE is a problem. Or deny that there are some unfortunate conflicts of interest between commercial parties, committers, reviewers and the community. Or deny that the FreeBSD community lacks the people, the finances or the time to do as much as it would like.

You can also try and turn this into an ‘OpenSSL moment’, a code quality disaster that became a turning point. One that will make the corporate giants that depend on FreeBSD realise that they have mostly been getting something for nothing and that it’s in their interest to make sure that the FreeBSD review process is in good health. Perhaps it requires more than just an occasional financial donation to the Foundation, perhaps donating staff time to have some of their developers do a couple of hours of ‘review service’ a month is more helpful.

I prefer to see it as the latter and hope FreeBSD comes out better from this. Hopefully a year from now we can look back and see that the project needed this crisis to come out stronger.

 

[drhowarddrfine]

Morris None of that has anything to do with the issue except it was a discovery of a failure of process (and not bad code or anything else). FreeBSD has been doing quality, commendable work for over 25 years but some are wanting to pretend this one ugly bug in that process disqualify everything accomplished so far. To use this as a reason to disrespect all this work is shameful, though my complaint is more against the writers who think along those lines.

 

[Trihexagonal]

team of professional, highly skilled psychologists to check the so-called soft skills, before granting a commit bit to a developer.

Yeah, and then immediately disqualify anyone the "highly skilled" psychologists approve.

Let's strike a happy medium. I volunteer.

My resume:

Psycho Psychologist
Is there any doubt of my qualifications?

Manipulative Bastard
Probably qualify as a Magnificent Bastard but I like to keep it on a personal level..

Morally Ambiguous Doctorate
Is that going to ba a problem? I can never tell.

Xanatos Speed Chess
Anything you say can and will be used against you.

I believe my work speaks for itself.

 

[Mjölnir]

Morris None of that has anything to do with the issue except it was a discovery of a failure of process (and not bad code or anything else).

Actually, it seems to be also about code quality.

FreeBSD has been doing quality, commendable work for over 25 years but some are wanting to pretend this one ugly bug in that process disqualify everything accomplished so far. To use this as a reason to disrespect all this work

IMHO that is your personal feeling. Where can I read that in the Ars Technica article? You keep denying that this incident revealed a serious underlying problem.

 

[Zirias]

You keep denying that this incident revealed a serious underlying problem.

Yes, and that should not be denied, otherwise, there might be a next time, and there might be less luck involved.

But: the article assumes/suggests the problem is a structural one. We can't know that. It should be analyzed, of course.

 

[eternal_noob]

the article assumes/suggests the problem is a structural one. We can't know that.

Of course it is a structural problem if a committer can close any review as "not accepted" on his own.

 

[Zirias]

Of course it is a structural problem if a committer can close any review as "not accepted" on his own.

I don't think so. You don't have to enforce every rule technically. It should of course ring alarm bells when something like this happens. So, it didn't (immediately) in this case, but that doesn't necessarily mean the problem is structural. It could be a hint, but the team has to analyze that…

 

[Jose]

I have to say that I see some quite unhelpful ad hominem attacks on the messenger here.

Oh, the irony!

Now, you can do two things...

I reject your false choice. The first Ars article on the subject was good and informative. The second one is a pure exercise in muckraking and concern trolling. Salter delivers a clinic on the latter here:

FreeBSD is an important project that deserves to be taken seriously. Its downstream consumers include industry giants such as Cisco, Juniper, NetApp, Netflix, Sony, Sophos, and more. The difference in licensing between FreeBSD and Linux gives FreeBSD a reach into many projects and spaces where the Linux kernel would be a difficult or impossible fit.

And the muckraking takes the form of splashing the seamy details of Macy's criminal past all over the Internet. And I'm supposed to worry about ad hominem attacks against Salter? Please.

He and Ars saw a way to turn a quick back by publishing all the saucy details of this melodrama, and to score some points with the Linux fanboi crowd by muttering darkly about what this must say about the quality of the Freebsd project overall.

FOSS development is messy sometimes. All kinds of soap operas can be written about it. I'm sticking to my guns. The process worked. No bad kernel-mode code was released. We're going to get a high-quality in-kernel Wireguard implementation, and the code review process is going to get some love.

I reject the Freebsd is doomed nonsense. Go read early 2000s Slashdot if you need a shot of "BSD is dying!"

 

[drhowarddrfine]

You keep denying that this incident revealed a serious underlying problem.

So my calling it an ugly bug in the process that needs fixing is denial? I've said, perhaps three times in this thread that it's a problem with the process.

 

[grahamperrin]

… the lowest form of person. Such as … typical posters on reddit.

That's certainly not true of /r/freebsd.

I spend far more time in Reddit than here because sadly, discussion of -CURRENT is forbidden.

 

[Snurg]

That's certainly not true of /r/freebsd.

I can confirm this, having read some threads there.

I spend far more time in Reddit than here because sadly, discussion of -CURRENT is forbidden.

Really, forbidden?

 

[grahamperrin]

Topics about unsupported FreeBSD versions

The FreeBSD Forums cater primarily to end-users and systems administrators. As such, the Forums focus almost exclusively on FreeBSD versions that are officially supported according to the official FreeBSD website. Since resources are scarce, the FreeBSD Forums strongly suggest that anyone asking...

forums.freebsd.org

 

[Mjölnir]

I can confirm this, having read some threads there.

OOH, good to know, but OTOH, I feel strong reluctance even to read that site to check your findings

Really, forbidden?

I'm not a mod, but I'd strongly guess when you post about CURRENT on-topic (or in "off-topic"), noone will complain. IIUC the forum's goal is to provide a helping hand to users, and naturally that's not about developer's revisions -- the mailing lists are for that. STABLE is allowed, and even then, you're kindly requested to @least read/follow the respective mailing lists.