el_programmer

New Member


Messages: 4

Developers of a static analyzer decided to recheck the FreeBSD project and to show that even in such serious and qualitative projects PVS-Studio easily finds errors. This was the topic of their blog post - https://www.viva64.com/en/b/0496/. Although this project is regularly checked by Coverity, it had a considerable number of potential vulnerabilities (CWE).
image1.png


As the author states, he spent just 2-3 hours looking for the potential vulnerabilities, but it took him about 3 weeks to put it all in an article. The reason for the delay was that PVS-Studio issued a great number of general-analysis warnings, which had to be handled by doing certain settings of the analyzer. The vast majority of false positives appears because of various macros and they can be easily eliminated by using a variety of mechanisms, provided by PVS-Studio. Further on, the author gives examples of the tool customization with examples.

Besides the standard Copy-Paste errors and typos there were also 56 potential vulnerabilities detected. Of course, only a few of the found CWE errors can turn into CVE (more details about the difference between CWE and CVE can be found here https://www.viva64.com/en/b/0486/ ). However, the more bugs that fall under the classification of CWE are found by static analysis, the better.

There is no doubt that the use of the static analyzer will be much higher if it is used regularly, not occasionally. A one-time check, like the one the author described in the article, can be a good way of showing the abilities of the analyzer, but it won’t be of real use to the project. The whole point of static analysis is that a lot of errors can be corrected at an early phase of the development. Additionally, it is much easier to keep the analyzer report “clean” and not to look for errors among hundreds of false positives. Here we have a complete analogy with the compiler warnings.

P.S. This is the second analysis of FreeBSD using PVS-Studio. The report of 2016 can be viewed by the link - https://www.viva64.com/en/b/0377/ .
 

ShelLuser

Son of Beastie

Reaction score: 2,124
Messages: 3,797

Pretty lame post in my opinion, and I'm not just saying because I happen to prefer FreeBSD but because it has "spam" and incompleteness written all over it. And in my personal opinion it's also plastered with a serious dose of lameness.

For starters: the article doesn't even differentiate between local exploits and remote exploits, and that is a seriously important difference where security is concerned.

Second: It doesn't bother to mention which version of FreeBSD was used. Considering the amount of users who apparently think that STABLE and CURRENT are legit FreeBSD releases I wouldn't even be surprised if the author ran this on a developer snapshot and is now surprised to find bugs in it. Yeah, I wonder how that happened :rolleyes:

Third: It doesn't bother to tell us exactly where it found the code snippets. It just quotes code snippets leaving us to wonder where it's from. I can't be bothered to spend too much time on things which I deem lame, but at this point I wouldn't be surprised if I couldn't find those snippets anywhere myself (but that's just an assumption of mine).

Fourth (adding up to three): Most security suites, products and companies try to make the world a safer place. Leaving out crucial information (see point 3) doesn't help, and that makes it obvious what's really on the agenda here.

Fifth: Ever bothered to read up on the FreeBSD ports collection? Instead of sharing all this stuff which I personally perceive to be lame nonsense you could also have opted to get this software added to the ports collection. That would have made a much better push than something like this.

As always, just my 2 cents. But at this time I wouldn't even go near this software anymore.

(Update / Edit)

What do you know, I was right. So I read through that blog post of his and found a link to (I quote): "report about the check of FreeBSD project in 2016", found here. And that page has a very interesting quote:

The source code was taken from GitHub branch - 'master'.

In case you're wondering, this GitHub repository. And what version did he get? Just as I suspected above: he grabbed CURRENT, a developer snapshot which is known to be buggy and doesn't even provide any guarantees to work at all.

So there's nothing to see here.
 
OP
E

el_programmer

New Member


Messages: 4

Pretty lame post in my opinion, and I'm not just saying because I happen to prefer FreeBSD but because it has "spam" and incompleteness written all over it. And in my personal opinion it's also plastered with a serious dose of lameness.

For starters: the article doesn't even differentiate between local exploits and remote exploits, and that is a seriously important difference where security is concerned.

Second: It doesn't bother to mention which version of FreeBSD was used. Considering the amount of users who apparently think that STABLE and CURRENT are legit FreeBSD releases I wouldn't even be surprised if the author ran this on a developer snapshot and is now surprised to find bugs in it. Yeah, I wonder how that happened :rolleyes:

Third: It doesn't bother to tell us exactly where it found the code snippets. It just quotes code snippets leaving us to wonder where it's from. I can't be bothered to spend too much time on things which I deem lame, but at this point I wouldn't be surprised if I couldn't find those snippets anywhere myself (but that's just an assumption of mine).

Fourth (adding up to three): Most security suites, products and companies try to make the world a safer place. Leaving out crucial information (see point 3) doesn't help, and that makes it obvious what's really on the agenda here.

Fifth: Ever bothered to read up on the FreeBSD ports collection? Instead of sharing all this stuff which I personally perceive to be lame nonsense you could also have opted to get this software added to the ports collection. That would have made a much better push than something like this.

As always, just my 2 cents. But at this time I wouldn't even go near this software anymore.

(Update / Edit)

What do you know, I was right. So I read through that blog post of his and found a link to (I quote): "report about the check of FreeBSD project in 2016", found here. And that page has a very interesting quote:



In case you're wondering, this GitHub repository. And what version did he get? Just as I suspected above: he grabbed CURRENT, a developer snapshot which is known to be buggy and doesn't even provide any guarantees to work at all.

So there's nothing to see here.




Firstly, PVS-Studio doesn’t only point to the bugs in the code. As we don’t know about the code as deeply as the code authors, we cannot say for sure about local and remote exploits. The main thing is that the tool found some errors that require fixing.

Secondly, we took the latest version of the project from GitHub.

Thirdly, of course it wouldn’t be very convenient to fix errors just by looking at the article, but it is also written that we are ready for the cooperation with the FreeBSD developers. The easiest variant is that we can give them a full report, which they will be able to convert using the utility Plog Converter: https://www.viva64.com/en/m/0036/#ID0EUIAI. In the article itself you may see the names of the files and lines.
 

bsdimp

New Member

Reaction score: 39
Messages: 15

Most of the things found should be fixed. However, their impact is overblown by lack of careful analysis. Many of the cases he's complaining about are cases where the tests against null should simply be removed or turned into asserts because they 'can't happen' because the variables are assigned at object creation. The memset in the kernel thing is a non-issue because the kernel is a stand-alone environment and we specifically disable many optimizations that are not appropriate for the kernel. The whole tone of the article, coupled with such rookie mistakes makes it difficult to approach with an open mind as a serious source of useful information. If you'd like to work with FreeBSD developers, that's better done through private communications to get the problems fixed rather than trying to make a big splash with an article with misleading headlines that only trigger a defensive reaction.

I agree that the tool found things that should be fixed. It's the implications that they are all exploitable I take issue with. There might be 1 in the whole bag of 56, and even that one I don't think can be exploited.

Since I am a FreeBSD developer, how about we work together to get these issues fixed rather than us working from a sensationalized article that's difficult to mine actionable details from.

Warner
 

OJ

Daemon

Reaction score: 401
Messages: 1,255

Secondly, we took the latest version of the project from GitHub.

That's not the version you should look at. How do you know some of these bugs haven't been fixed or simply don't exist in the non development version that we normally use?
 

tingo

Son of Beastie

Reaction score: 653
Messages: 2,566

So just a way to generate interest for a new product. As has been said already in this thread - nothing to see here.
 

Phishfry

Beastie's Twin

Reaction score: 2,872
Messages: 5,871

I consider it a good thing whenever someone attempts to help people for free.

If you look at the blog post he offers a free license offer. While distasteful for some using a paid program for fuzzing FreeBSD overall it seems like a good thing. Looking at the 2016 scan I see several coding errors that were caught. While there does seem to be many false positives I think as a user this is a good thing. More eyes on the code. We are all after the same thing. A more secure FreeBSD.

I could see where a developer might tale offense. They might feel that everything they do is perfect and challenging their assumptions is an insult. I do it in the machine shop with our QA department and I could understand a coder might too. Especially being a volunteer.
Being a creator you tend to become zealous of your work.
 

Phishfry

Beastie's Twin

Reaction score: 2,872
Messages: 5,871

Just to play devils advocate I see lots of posts complaining about using -CURRENT to test with.
If you read his blog I think its quite clear.

"The whole point of static analysis is that a lot of errors can be corrected at an early phase of the development."

FreeBSD -CURRENT being the development version.
 

ShelLuser

Son of Beastie

Reaction score: 2,124
Messages: 3,797

I could see where a developer might tale offense.
The only thing I take slight offense to is the sensational headline of that blog post. He makes it sound as if he found 56 potential vulnerabilities in one evening while that simply isn't true. Plain and simple. He also indirectly implies that the FreeBSD developers could have been using his tool to prevent all these "potential vulnerabilities", while in fact most of what was found were false positives and it was highly questionable if the other things could actually be exploited at all (also see the links in SirDice post).

It's not just about the message, but also how you deliver it.

(Edit)

And since we're now playing the devils advocate... This software only has 2 licenses: A team, and an enterprise. No free or community licenses available. That's one major negative aspect here because now it's obviously just about selling this software, nothing more.

And to add insult to injury the team license is only applicable for the Windows version. So how is that useful for a single FreeBSD user/programmer I wonder?

Also note that this software doesn't actually provide a BSD version at all. There's only a Windows and Linux version. Now, I know FreeBSD has linux compatibility layers but surely... Still, this is about trying to sell Linux software to FreeBSD developers while only providing enterprise pricing schemes.

... while listing dozens of false positives and claiming you found 56 potential exploits.

Disclaimer: I'm not a FreeBSD developer, but still got annoyed with the post nonetheless.
 

forquare

Well-Known Member

Reaction score: 187
Messages: 337

I like to have things looked at by outside eyes. Outside QA is painful but necessary.

I would agree with that, but perhaps the correct way to go about this would have been to have looked over the bugs and submitted bug reports (optionally with patches). Of course that might have hurt the sales pitch…
 
OP
E

el_programmer

New Member


Messages: 4

Second: It doesn't bother to mention which version of FreeBSD was used. Considering the amount of users who apparently think that STABLE and CURRENT are legit FreeBSD releases I wouldn't even be surprised if the author ran this on a developer snapshot and is now surprised to find bugs in it. Yeah, I wonder how that happened :rolleyes:

We have rechecked a fresh version of the FreeBSD code using PVS-Studio. Git revision: 59fe28863e6a0903b50b37c616f21a2a865bbbf2

We have worked on the reports a bit, having filtered those messages that seemed unnecessary. There are some other false positives in the list of course, but it’s not possible to eliminate unnecessary warnings in large groups. The remaining warnings should be reviewed separately.

The report is provided in two formats (tasks and csv). To those who will start working with the report: perform the automatic replacement of SOURCE_ROOT with the necessary path, so that the navigation works well.

Tasks: http://cppfiles.com/freebsd.plog.tasks

Csv: http://cppfiles.com/freebsd.plog.csv

We are ready to help and answer any questions.
 

ShelLuser

Son of Beastie

Reaction score: 2,124
Messages: 3,797

How to use PVS-Studio for Free - https://www.viva64.com/en/b/0457/
I noticed, and despite my previous criticism I do applaud you for setting that option up. However the one thing which still puzzles me and also led up to my previous posts: why isn't this mentioned on your FAQ nor on the ordering page?

It's one thing to add a seclusive option which allows people to use your software for free, it's another to fully stand behind it.

Fact is that if you follow the products mainstream link you won't come into contact with this information. Even when people follow your FAQ entry on "Your prices are too high" you don't tell them anything about this option. So yeah... My previous opinions on motivation still stand.

We have rechecked a fresh version of the FreeBSD code using PVS-Studio. Git revision: 59fe28863e6a0903b50b37c616f21a2a865bbbf2
You really like to obscure things, do you? I suppose I would too if I wanted to cause a little bit of confusion.

Sure you don't meant to say 20170407 (most current revision at the time of writing)?

So the thing is: the revision isn't the issue here, the branch is. Since it seems clear to me that you don't want to put any effort into reading up on how this works let me spell it out for you: FreeBSD knows officially supported releases, as well as developer snapshots where the latter are not meant for production and are therefor also not supported to be used in such a way. Quite the contrary even; there are no guarantees given that things will work at all.

And here you are: pulling out such a developer snapshot, which includes tons of debug code and all, and you're flaunting yourself that your software found possible exploits. No shit. I'd honestly be surprised if it didn't.

Now, I do agree: the descriptive names can be confusing. I mean... Naming unstable developer snapshots CURRENT and STABLE? I can definitely understand where the confusion comes from. However... If you take the 3 minutes effort to actually read the release webpages linked by me above you'll soon discover this for yourself.

(edit) retracted unneeded comment regarding webpage reading.

Anyway.. maybe you should try to look into the release/11.0.1 branch, or commonly put: FreeBSD 11.0-RELEASE-P1. Or put differently: 20160928. Otherwise you're merely discrediting your own efforts here.

PS (edit):

Why do you tell people that the prices can be found on the ordering page, while in fact they're not:

Code:
To order the license and get the pricing information, please contact us.
With all due respect: that does not look like an honest and transparent sale at all. What guarantees do we have that you won't charge a fan of yours less than a critic for example?

Honestly, this is not a good way to try and appeal to open source communities. In my opinion of course.
 
  • Thanks
Reactions: OJ

Pegasus711

Member

Reaction score: 6
Messages: 74

I am not a FreeBSD developer but nonetheless his initial posts seemed a bit sensationalist but still I believe it wouldn't hurt to have such analysis done before at least CURRENT becomes STABLE. It wouldn't hurt.

Some of the stuff like de-referencing NULL pointers even if never encountered in real life gives a bad image of such a high quality project which is used as reference learning material (along with other kernels like Linux) in quite a few schools and texts.

Just my two cents. I hope you fellas take that in the right spirit. One of the reasons I have tilted more towards the BSDs specifically FreeBSD compared to Linux is because quite a few folks there lack some humility (or at least it looks so when looking at some of the comments made by their top guys. Torvalds have set a really bad precedent in that respect). I hope we don't go the Linux way in that sense
 

Phishfry

Beastie's Twin

Reaction score: 2,872
Messages: 5,871

Even thought this was a great thing I think that a forum post is not the proper channel. This needs to go to the proper people. Not the forum.
This is the place for people with little problems.
 
Top