Update 10am June 21, 2011: One press release so far.
Update 2, 10am June 21, 2011: Nigel Dessau has a blog up. This contains the tip of the iceberg for bitching. Others have much more to say. Rest assured, it will come out.
BAPCO has turned into a bad joke, so bad that Nvidia (NASDAQ:NVDA), AMD (NYSE:AMD), and VIA (2388:Taiwan) just quit. Yes, that leaves Intel (NASDAQ:INTC) as the only semiconductor maker still in the consortium, and there is a good reason for this.
The short story is this, BAPCO makes the SYSmark series of benchmarks, and several others. SYSmark in particular has been long seen by everyone but Intel as a joke, it is so skewed that it is laughable, it really doesn’t measure anything but Intel or non-Intel CPUs. With the release of SYSmark 2012, things got even worse.
SemiAccurate has been hearing rumblings about discontent from everyone involved in BAPCO, everyone but Intel of course, about how broken SYSmark 2012 is. The complaint, again from multiple vendors, is that Intel owns the process, and overrides anyone’s views, thoughts and additions.
We brought these concerns up to a few Intel folk at Computex a few weeks ago. Talking, unofficially because of confidentiality agreements, we heard the same tired refrain, that the process is open and any member can submit anything for a vote. True enough. The problem is that every time someone does, it gets shot down, unless that someone is Intel. Their wishes seem to get in every time for some odd reason.
For months now, the rancor over SYSmark 2012 has grown to a fever pitch, complaints are duly put forward at meetings, and duly ignored. One of the biggest is that the benchmark completely ignores GPU power, if you have a barely functional Intel IGP and add a GTX580, your score won’t budge. Go from a dual core to a quad, and watch the numbers jump. This is a regression, a bad one from the last version. Any guesses who benefits from this little quirk, and who loses? Try it for yourself and see.
There are lists of others, and we have been read bits of several rancorous letters, and told more that were seen as too futile to bother bringing up. The list of fatal flaws with SYSmark 2012 is long and ugly, so long as to render the suite pointless for any real use. And nothing was done about it despite protests. Rather than lend their names to such a sham, Nvidia, AMD, and VIA all quit, leaving Intel to dominate all areas. The status quo is sadly unchanged.
Ironically, if you look at either the about us page or the news page, there is no mention of 3/4 of the chip companies that were members quitting. How odd. Granted, it won’t change the voting practices, but you have to admit, it does look bad. </sarcasm> This looks worse though. </no sarcasm>
Why does Intel do this? Why do they deny it so vocally? Because SYSmark is a widely used sales tool, especially in government and large corporations, and it is supposed to be an open industry benchmark. For it to be seen as anything less would taint the process, so those with vested interests have to shout, and shout they do. Luckily, that pesky FTC thing is settled, otherwise there might be some nasty questions asked.
In the end, BAPCO and SYSmark 2012 is now an official shame, not just one in name only. Anyone using it seriously should be immediately suspect for both motives and technical awareness. If a salesperson comes to your company and mentions the suite, you know who they are pushing, it is that bad. For some reason, BAPCO itself is going to great lengths to keep all this quiet, I wonder if that was an open vote too? One thing for sure, the silence is deafening.S|A