What do you guys think about this benchmark
http://www.phoronix.com/scan.php?page=article&item=intel_snb_freebsd&num=4
http://www.phoronix.com/scan.php?page=article&item=intel_snb_freebsd&num=4
Bobbla said:I just read the first page and looked at the pictures on the rest...
I've got to ask, it seems like that graphics card is the ONLY hardware that is "equal" for both OS's. So how can this be a fair test if you are testing OS's on different hardware? Shouldn't the hardware be the same if you are testing the OS?
I'm confused... the author couldn't have just neglected this? right? :\
I don't understand how benchmarking an operating system in a state that it would never normally be used is of any relevance. Why do we care how operating systems perform in single user mode?phoenix said:The only way to make this benchmark relevant would be to:
- Install FreeBSD onto the laptop, without any GUI. Do the benchmarks from single-user mode (or at last with as many services disabled as possible) several times. Graph the results.
- Install Ubuntu onto the laptop, without any GUI. Do the benchmarks from single-user mode (or at last with as many services disabled as possible) several times. Graph the results.
aragon said:I don't understand how benchmarking an operating system in a state that it would never normally be used is of any relevance. Why do we care how operating systems perform in single user mode?
aragon said:I don't understand how benchmarking an operating system in a state that it would never normally be used is of any relevance. Why do we care how operating systems perform in single user mode?
Different hardware? Where?alie said:they are comparing 2 different OS with different hardware...
alie said:Another bad benchmarking... i am speechless hahaha... they are comparing 2 different OS with different hardware...
http://www.phoronix.com/scan.php?page=news_item&px=MTAyNzA
On 12/15/2011 04:41 AM, Michael Ross wrote:
> Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel
> <michael.larabel@phoronix.com>:
>
>> On 12/15/2011 02:48 AM, Michael Ross wrote:
>
>>> Anyway these tests were performed on different hardware, FWIW.
>>> And with different filesystems, different compilers, different GUIs...
>>>
>>>
>>
>> No, the same hardware was used for each OS.
>>
>
> The picture under the heading "System Hardware / Software" does not
> reflect that.
>
> Motherboard description differs, Chipset description for FreeBSD is
> empty.
>
I was the on that carried out the testing and know that it was on the same system.
All of the testing, including the system tables, is fully automated.
Under FreeBSD sometimes the parsing of some component strings isn't as nice as Linux and other supported operating systems by the Phoronix Test Suite. For the BSD motherboard string parsing it's grabbing hw.vendor/hw.product from sysctl. Is there a better place to read the motherboard DMI information from?
-- Michael
frooyo said:Has there been any talk of someone reproducing the test results.
Also, you mentioned this topic has been talked about a lot on the mail list. Where can I read that conversation on the web?
wblock@ said:If so, I missed it. I think the benchmark is downloadable, but unless you have the same hardware... or are planning on duplicating both their Linux and FreeBSD tests. If you are planning on doing a useful benchmark, it would be good to ask for suggestions first. It helps to be thick-skinned, as benchmarks are easy to do wrong, hard to do right, and generally a boring, painstaking, thankless chore. Science, in other words.
Much of it is here: http://lists.freebsd.org/pipermail/freebsd-current/2011-December/thread.html. Search for "benchmark".
wblock@ said:See, that's the tricky part. Different hardware might have vastly different reactions to the same code. CPU and disk cache, bus bandwidth, there are thousands of things that would be changing. Running in a VM is a big difference, too. If you can remove all the variables except one--or in reality, a few--then you can get meaningful comparisons.
If you run the same benchmarks in a VM and FreeBSD is vastly faster... what could you point to and say "My test shows the opposite of theirs, and here's why mine was valid."