Speed Test 2: Comparing C++ Compilers On WIndows

Nerval's Lobster writes "In a previous posting, developer and programmer Jeff Cogswell compared a few C++ compilers on Linux. Now he's going to perform a similar set of tests for Windows. "Like all things Windows, it can get costly doing C++ development in this environment," he writes. "However, there are a couple notable exceptions" such as free and open-source cygwin, mingW, Express Versions of Visual Studio, and Embacadero. He also matched up the Intel C++ Compiler, Microsoft C++ Compiler, and the Embarcadero C++ 6.70 Compiler. He found some interesting things — for example, Intel's compiler is pretty fast, but its annoying habit of occasionally "calling home" to check licensing information kept throwing off the rests. Read on to see how the compilers matched up in his testing."

Calling home

[six025]

>> its annoying habit of occasionally "calling home" to check licensing information

Calling home for the latest NSA exploits to inject in to your application? /tinfoil-hat-no-so-paranoid-these-days-dept

Calling home threw off the results?

[iYk6]

Did calling home really throw off the results? Since that is something that ordinary users would have to put up with, I would think it should be part of the test. It might be difficult to get an average, but testing Intel's compiler only when it is at its fastest doesn't seem fair.

Generated code speed

[c++]

This doesn't test the speed of generated code. I like to know which compiler produces faster code when looking at benchmarks.

Inaccurate test.

[johnnys]

According to the fine article, "The Intel compiler occasionally âoecalls homeâ to an Intel-owned Website to check licensing information. When it does so, it prints out a message about when the current license expires. I didnâ(TM)t use the results when that happens, since it would add time and skew the timing results. " WRONG. The tester should not have excluded these results where time was wasted with this nonsense: If WE the users have to put up with it, it SHOULD be included in the benchmarks.

Useless Comparison

[Anonymous Coward]

Wow, lets look at what's being measured here, COMPILE TIME, and EXECUTABLE SIZE...what about the performance of the generated application.

This doesn't measure optimizations, this just measures COMPILE TIME..I don't care if my applications takes 1 sec or 1 hour to compile, I care about the PERFORMANCE of the actual APPLICATION.

This is just crap.

Re:Representative benchmarks?

[c++]

In other words, I have no reason to assign any meaning to these numbers.

Given the reaction to the previous article I don't know what this guy is even trying to do.

And why 6084? What is so special about that number?

6084 / 2 % 100 == 42

That is meaning enough.

Re:Useless Comparison

[Anonymous Coward]

Wait until you grow up and are compiling systems with millions of lines of code that takes hours. We were happy when a compiler upgrade cut our times from 10 hrs, to 5. You may not care for your kiddie projects, but in the real world, time is money.

Re:Inaccurate test.

[TheCarp]

while absolutely correct, and not just we put up with it.... if the license check is what the compiler does, then it is what it does. To leave those out is to be measuring something other than the real behaviour of the compiler in real situations.

Hell if this is the case, can you really call the testing complete if he didn't simulate network conditions like, the licensing server being unreachable, or having really high latency?

Re:Representative benchmarks?

[Ravaldy]

The article is alright but not one I would use to pick a compiler. IMHO the resulting EXE is more important than the compiler processing time. I've dealt with large sized applications and if structured properly, your build times on a modern computer should not be an issue.

Re:Crickets...

[NotBorg]

It would help if he actually measured something worthwhile. In the 20+ years I've been coding, I've never once picked a compiler on the basis of how long it takes to spit out a binary. There are just so many other more interesting features and characteristics to consider.

Invalid Benchmark - Who Cares

[MerlynEmrys67]

Benchmarking compilers on how long it takes to compile would be like benchmarking cars based on how long it takes to fill the gas tank.
There are so many things that can affect compile time more than the compiler - and the end customer really doesn't care anyway. Frankly, if you want a 3-5x speedup, just put the whole thing on an SSD and let it fly.

Re:does the Intel one still slow down on AMD syste

[fuzzyfuzzyfungus]

If memory serves, their argument varies (depending on whether the FTC appears interested or not) between 'fuck you, it's the Intel compiler collection, and it'll do what's best for Intel. Go suck an Opteron if you like AMD so much.' and 'Gosh, we sure know about the capabilities flags; but we can't be sure of the details of other vendors'(*cough*shoddy, probably reverse engineered illegally*cough*) implementations of certain complex features, and our customers expect our compiler suite to provide stable, correct output, so reverting to the x87 codepath is our only real option..."