JBQ's spot on the Wild Wild Web
The musings of a French mathematician living in the heart of the American technology industry

Benchmarks

Ahhhhh, benchmarks. An easy way to see if people say the truth or not when they speak about the respective speeds of their CPUs. I like to see what people say, and compare that to SPECCPU2000 results. The differences are sometimes very surprising. Well, are they so surprising? Let's have a look at the 3 major "vendors" of CPUs systems, Intel, AMD and... Apple (because Motorola doesn't seem to gloat about the performance on the PowerPC G4, only Apple does).

AMD has recently released their new Athlon XP 2200+. Is it really faster than a 2200 MHz CPU? On integer stuff, the AthlonXP is good for 738 points. The funny thing is, a Pentium4 at a mere 2GHz scores the same 738 points. Oh, yes, I know, that's because AMD has a superior floating-point performance. Sure. CFP2000, AthlonXP goes as high as 624 points. And the poor little Pentium4 at 2GHz with its slow FPU only gets 744 points. Please read that again. So, how much floating-point power is there in an Athlon XP 2200+ running at 1800 MHz? Well, about as much as in a Pentium4 running at 1600MHz. Man I wouldn't want to have just read that if I was an AMD zealot, that's gotta hurt.

Don't worry, my AMD friend, your CPU performs more than adequately. Wait until I talk about the "super-computer" G4 that is used by Apple.

G4 1000MHz: 306 points in integer. Just like a PIII at 667MHz. But, as you all know, The G4 is extremely good in floating point, capable of doing billion operations per second. G4 1000 MHz: 187 point in floating point. That's the level of a PIII at 500MHz. Oh my God, if I overclock three-year-old my dual-PIII from 450 to 504 MHz (where it is perfectly stable), I get as much FPU power as a top-of-the-line Mac. I don't know if I should laugh or cry. I just feel sad for all the people who fall in for Apple's propaganda. If a Mac can do all that a "Wintel" PC can do (yeah, right), well, it'll be doing it much much much much slower.

A few comments before people flame me. Or maybe a few comments that'll cause people to flame me...

I picked the baseline results over the peak results. Because I only had baseline results for the G4, and because I think that they are more realistic to show real-world speed: if you're a developer, just use the same compiler flags as Intel, Dell or AMD used, they are published in the benchmark report.

The fact that the G4 benchmarks come from a magazine and are not official results. I would normally have put a disclaimer about that. Well, if you're not happy about the results, please go and put some pressure on Apple to publish official results. I monitor the SPEC results on a regular basis, and I'll be more than happy to take any official results into account.

Some zealots will say that the G4 can do better than that because gcc doesn't use Altivec. Well, now, it's not my fault if you don't have a decent compiler, is it? Do you think that someone with a mind would go spend some time hand-optimizing his/her code in assembly for a CPU that only has a few percent of market share? Imagine a team of 30 engineers trying to release an application simultaneously for Windows and MacOS. 28 engineers write the portable core of the application (and they all develop on Windows with Visual C++ and Purify), 1 engineer is responsible for the Windows adaptation layer and Windows optimization (like, tweak the compile flags for the intel compiler), 1 engineer is responsible for the MacOS adaptation layer, MacOS-specific issues and MacOS optimization (like, deal with a compiler that doesn't support the Visual C++ extensions, deal with a CPU that orders bytes differently, deal with an OS that'll do some things differently, like not have drive letters, use slashes instead of backslashes as a file separator, not support MDI, put the menubar at that top of the screen, and when there's a little bit of time left, re-write in assembly a routine that the original programmer will modify so much before the release date that it'll have to be re-written in assembly 5 times in the coming year). I wouldn't want to be the MacOS guy.

Oh yeah, I've also read that running SPEC benchmarks for PowerPC was unfair because the benchmarks are x86-specific. Well, I guess that the same benchmarks are also unfair for HP-PA CPUs, Itaniums, Sparcs, MIPS, Alphas, POWER... which all manage to beat the G4. The only reason why they're "unfair" for PowerPC is that those benchmarks are written in C, C++ and Fortran, and that the measure as much the compiler as the CPU. Got a sucky compiler? You'll get bad SPEC results. Guess what? Got a sucky compiler? You'll get bad results on everything but the 3 routines that Apple will optimize by hand to make Altivec shine...

So, I guess that the only CPUs worth considering are Intel's. The Pentium4 rocks (I'm dreaming of buying a dual-Xeon 2200 MHz, but I'm not sure I want to afford the $2000 that such a beast costs). Itanium looks very promising, especially with an incredible FPU power (if you want numbers, well, how about 645 points on last year's hardware?).

Home page Related articles Posted on Jul 23 2002