Hello there,
I just got my new motherboard in the mail (the old one was shot and couldn't boot FreeBSD), and I put FreeBSD-9.0-CURRENT (201105-amd64-dvd1) on my box to try to figure out a lingering question I had... should I use clang or gcc? So, I did some tests!
First of all, I pulled FreeBSD head Rev 224044 for the tests, and built (worlds only, no kernels) one system with gcc (4.2.2-20070831), and the other with clang (3.0-20110502)and installed them in a chroot.
Note: All tests were done in RAM (the files to tar, the tarred file, and the flops program, but the system/world was on the HD). Also, the kernel used was compiled with gcc, I'll release benchmarks with a clang kernel later. Given that, these results are still very valid
System specs:
Intel i7-970 (6-core with HT [12 threads] @ 3.20GHz)
24GB ram (DDR3-1600 [Damnit, I have it set as 1067MHz in the BIOS])
50GB SSD HD (200MB/s write, 250MB/s read)
Test #1: tar.bz2ing the FreeBSD head (cleaned) that I used to build the systems
Note: I tarred the files first, then tarred again with timing to clear up having caching changing performance a lot
`tar cjf test.tar.bz2 tarme/`
`time` on clang:
209.75 real 203.81 user 5.93 sys
`time` on gcc:
215.62 real 207.54 user 8.07 sys
Test #2: md5ing the generated tarball 100 times
`time` on clang:
79.95 real 46.29 user 33.65 sys
`time` on gcc:
84.11 real 50.77 user 33.33 sys
Test #3: sha1ing the generated tarball 100 times
`time` on clang:
112.50 real 78.53 user 34.04 sys
`time` on gcc:
102.64 real 69.62 user 33.02 sys
Test #4: sha256ing the generated tarball 100 times
`time` on clang:
196.92 real 163.53 user 33.38 sys
`time` on gcc:
204.38 real 171.47 user 32.90 sys
Test #5: MFLOPS from the benchmarks/flops port
flops.c built as:
`clang -O2 -DUNIX flops.c`
`gcc -O2 -DUNIX flops.c`
NOTE: THERE ARE 2 OUTPUTS FOR EACH SYSTEM FOR REDUNDANCY
clang output #1:
clang output #2:
gcc output #1:
gcc output #2:
Average MFLOPS for clang:
1388.715075
Average MFLOPS for gcc:
1342.7099875
All in all, it seems that clang outperformed gcc by a decent bit when it came to tar, md5, sha256, and MFLOPS. The ONLY place where gcc seemed to outperform, was in sha1, which seems to be a fluke (bit I still double checked and it isn't).
When it comes to md5, sha256, and tar, there is a quite considerable difference, and in MFLOPS it does manage to beat gcc, but not by a very noticeable amount.
Now, you could go out and build a modern version of gcc which would probably outperform clang. But a modern gcc is not supported at all on FreeBSD and I'd prefer the BSD license over the GPL anyways.
Quick summary:
- If you care about performance, would sacrifice support, and don't care about licensing, use a modern day GCC.
- If you care about performance, want support (but can sacrifice a tiny amount of it), and care about licensing, use clang.
- If you don't care about performance, want the best support, and don't care about licensing, use the bundled 'old' gcc.
To me, the license and support pull me to clang, and as a dev I'm always looking for new projects to contribute to, and this one looks perfect for my tastes.
If you have any questions about my build environment, or want me to do other benchmarks (I probably wont do any more after tomorrow (Friday) as I'll be reinstalling a clean build of FreeBSD with clang).
Cheers,
Brandon Falk
I just got my new motherboard in the mail (the old one was shot and couldn't boot FreeBSD), and I put FreeBSD-9.0-CURRENT (201105-amd64-dvd1) on my box to try to figure out a lingering question I had... should I use clang or gcc? So, I did some tests!
First of all, I pulled FreeBSD head Rev 224044 for the tests, and built (worlds only, no kernels) one system with gcc (4.2.2-20070831), and the other with clang (3.0-20110502)and installed them in a chroot.
Note: All tests were done in RAM (the files to tar, the tarred file, and the flops program, but the system/world was on the HD). Also, the kernel used was compiled with gcc, I'll release benchmarks with a clang kernel later. Given that, these results are still very valid
System specs:
Intel i7-970 (6-core with HT [12 threads] @ 3.20GHz)
24GB ram (DDR3-1600 [Damnit, I have it set as 1067MHz in the BIOS])
50GB SSD HD (200MB/s write, 250MB/s read)
Test #1: tar.bz2ing the FreeBSD head (cleaned) that I used to build the systems
Note: I tarred the files first, then tarred again with timing to clear up having caching changing performance a lot
`tar cjf test.tar.bz2 tarme/`
`time` on clang:
209.75 real 203.81 user 5.93 sys
`time` on gcc:
215.62 real 207.54 user 8.07 sys
Test #2: md5ing the generated tarball 100 times
`time` on clang:
79.95 real 46.29 user 33.65 sys
`time` on gcc:
84.11 real 50.77 user 33.33 sys
Test #3: sha1ing the generated tarball 100 times
`time` on clang:
112.50 real 78.53 user 34.04 sys
`time` on gcc:
102.64 real 69.62 user 33.02 sys
Test #4: sha256ing the generated tarball 100 times
`time` on clang:
196.92 real 163.53 user 33.38 sys
`time` on gcc:
204.38 real 171.47 user 32.90 sys
Test #5: MFLOPS from the benchmarks/flops port
flops.c built as:
`clang -O2 -DUNIX flops.c`
`gcc -O2 -DUNIX flops.c`
NOTE: THERE ARE 2 OUTPUTS FOR EACH SYSTEM FOR REDUNDANCY
clang output #1:
Code:
FLOPS C Program (Double Precision), V2.0 18 Dec 1992
Module Error RunTime MFLOPS
(usec)
1 4.0146e-13 0.0080 1753.5920
2 -1.4166e-13 0.0065 1072.0893
3 4.7184e-14 0.0062 2756.7597
4 -1.2557e-13 0.0061 2453.0056
5 -1.3800e-13 0.0116 2506.2720
6 3.2380e-13 0.0272 1065.8150
7 -8.4583e-11 0.0257 466.0625
8 3.4867e-13 0.0120 2505.2298
Iterations = 512000000
NullTime (usec) = 0.0000
MFLOPS(1) = 1339.7506
MFLOPS(2) = 938.0637
MFLOPS(3) = 1508.7634
MFLOPS(4) = 1768.1651clang output #2:
Code:
FLOPS C Program (Double Precision), V2.0 18 Dec 1992
Module Error RunTime MFLOPS
(usec)
1 4.0146e-13 0.0080 1754.0005
2 -1.4166e-13 0.0065 1071.8402
3 4.7184e-14 0.0062 2755.8083
4 -1.2557e-13 0.0061 2450.9765
5 -1.3800e-13 0.0116 2505.0859
6 3.2380e-13 0.0272 1065.7021
7 -8.4583e-11 0.0257 466.5673
8 3.4867e-13 0.0120 2505.7709
Iterations = 512000000
NullTime (usec) = 0.0000
MFLOPS(1) = 1339.4153
MFLOPS(2) = 938.6299
MFLOPS(3) = 1509.0247
MFLOPS(4) = 1767.9079gcc output #1:
Code:
FLOPS C Program (Double Precision), V2.0 18 Dec 1992
Module Error RunTime MFLOPS
(usec)
1 4.0146e-13 0.0080 1754.2786
2 -1.4166e-13 0.0091 771.8205
3 4.7184e-14 0.0066 2566.9269
4 -1.2557e-13 0.0063 2396.7763
5 -1.3800e-13 0.0229 1268.5356
6 3.2380e-13 0.0111 2619.0838
7 -8.4583e-11 0.0240 500.9846
8 3.4867e-13 0.0232 1292.1545
Iterations = 512000000
NullTime (usec) = 0.0000
MFLOPS(1) = 1000.5765
MFLOPS(2) = 1009.2525
MFLOPS(3) = 1431.8632
MFLOPS(4) = 1929.1627gcc output #2:
Code:
FLOPS C Program (Double Precision), V2.0 18 Dec 1992
Module Error RunTime MFLOPS
(usec)
1 4.0146e-13 0.0080 1753.9284
2 -1.4166e-13 0.0091 771.5915
3 4.7184e-14 0.0066 2566.8247
4 -1.2557e-13 0.0063 2395.5376
5 -1.3800e-13 0.0229 1268.4892
6 3.2380e-13 0.0111 2619.6577
7 -8.4583e-11 0.0239 501.2391
8 3.4867e-13 0.0232 1292.0066
Iterations = 512000000
NullTime (usec) = 0.0000
MFLOPS(1) = 1000.3124
MFLOPS(2) = 1009.5551
MFLOPS(3) = 1431.9474
MFLOPS(4) = 1929.0101Average MFLOPS for clang:
1388.715075
Average MFLOPS for gcc:
1342.7099875
All in all, it seems that clang outperformed gcc by a decent bit when it came to tar, md5, sha256, and MFLOPS. The ONLY place where gcc seemed to outperform, was in sha1, which seems to be a fluke (bit I still double checked and it isn't).
When it comes to md5, sha256, and tar, there is a quite considerable difference, and in MFLOPS it does manage to beat gcc, but not by a very noticeable amount.
Now, you could go out and build a modern version of gcc which would probably outperform clang. But a modern gcc is not supported at all on FreeBSD and I'd prefer the BSD license over the GPL anyways.
Quick summary:
- If you care about performance, would sacrifice support, and don't care about licensing, use a modern day GCC.
- If you care about performance, want support (but can sacrifice a tiny amount of it), and care about licensing, use clang.
- If you don't care about performance, want the best support, and don't care about licensing, use the bundled 'old' gcc.
To me, the license and support pull me to clang, and as a dev I'm always looking for new projects to contribute to, and this one looks perfect for my tastes.
If you have any questions about my build environment, or want me to do other benchmarks (I probably wont do any more after tomorrow (Friday) as I'll be reinstalling a clean build of FreeBSD with clang).
Cheers,
Brandon Falk
