For our upcoming hardware reviews, we’ve spent some time rethinking the way we do our power testing. There are five major scenarios that describe how gamers (et. al.) use their PCs when it comes to power testing: desktop idle, in a shooter, in a strategy game, browsing social media, and watching video streams. Given these scenarios, the question becomes which apps specifically do we choose to compare power consumption?
What to Benchmark?
Desktop idle power consumption in Windows 10 is a straightforward matter. For a shooter, we want to pick something recent and popular but hopefully a bit forward-looking: DOOM (2016) comes to mind. Picking a strategy game is more complicated: I can think of two games that offer good benchmarks for looking at CPU performance: Civ VI and AotS. But which one to pick? In our testing, the Civ VI benchmark draws a maximum of 265 Watts (graph below) while the AotS benchmark draws a whopping 295 Watts (graph above) from the same system. Our goal here is to get as close to a worst-case scenario as possible inside of a real game.
For our browsing test, we turned to a benchmark designed to replicate the kind of load Facebook of Twitter create: Speedometer. Obviously, this is a bit condensed compared to the kind of endless scrolling through feeds found in most internet browsing scenarios. But it’s a very testable workload where differences in raw performance correspond to differences in user experience. By measuring the power consumed during these five scenarios we aim to create a more accurate representation of real world power consumption.
Measuring Power Consumption
As always we’re measuring power consumption at the wall rather than testing directly for CPU power consumption. We prefer this method because it puts the focus on what end-users will experience which isn’t the isolated power consumption of a specific component, but rather the effect of the whole system on their power bill. From a technical perspective looking at component specific power consumption is also rather difficult to do as a third-party.
Two potential methods are to use vendor specific power monitoring tools which required us to take the output of these tools at face value or to mod our motherboard to create testable points for data recording and increase the already high potential that magic smoke will escape from our test samples.
Thus when we do power testing what we’re looking at is platform power consumption. More importantly when we consider performance per watt in a specific benchmark we’re not looking at the perf/watt of a CPU architecture but rather of a real system based on that architecture and the support components it requires to be a functional gaming PC. The goal here is to produce information that’s relevant to the real-world rather than something esoteric. We’ll leave that business to first-party technical demos and the like.
A Grounded Way of Looking at Perf/Watt
This is differentiated from the performance per watt testing you’ll in other places around the web. Most of those comparisons are based on average performance divided by average load power consumption. Whereas we’re looking at scenarios to consider workload-specific performance and consumption and then generalizing from those reproducible scenarios. This gives us a lot more latitude to explore differences in performance per watt than a simple averages-based calculation.
UPDATE 2/20/2017 @ 10:32 PM CST: I’ve replaced the table below to clarify the dataset and I’ve added the test system information found above. The findings of this article have not changed.
To demo this scheme let’s compare two well-known CPUs: Intel’s i7-6950X and AMD’s FX-8370e. The support components in our testing will remain the same with only the CPU, motherboard, and memory changing between test runs.
For workloads where performance is measured through function, we’ve used 1 to represent adequate function. Averaged across all of these benchmarks the 6950K offers about 46 percent better performance per watt. Looking at the individual scenarios in these tables it’s apparent that there’s more to this story.
In some cases like our first person shooter scenario where the workload is largely GPU-limited performance per watt is similar. But in other scenarios like desktop idle and 4k YouTube playback, higher power consumption is enabled by the FX-8370e’s now ancient power management technology and even older platform chipset. Somewhat surprisingly the FX-8370e looks worst in our social media browsing benchmark. This is a workload that exploits the two major weaknesses of AMD’s old HEDT platform: weak single-threaded performance and high power consumption under load. These two factors conspire to create the performance per watt blowout we see above.
Ryzing Expectations for Ryzen
When it comes to performance per watt the stage is definitely set for the arrival of AMD’s Ryzen CPUs. A 46 percent performance per watt gap between Intel’s best HEDT CPU and the most efficient of AMD’s current offerings is a big gap to close. Over the past couple of years, AMD’s moved to designing its products as SoCs. They’ve also demonstrated a singular focus on power efficiency with Carrizo and Bristol Ridge in the mobile market. Thus it’s not difficult to image this performance per watt gap closing significantly. But still, AMD’s Ryzen certainly has its work cut out for it.S|A
Thomas Ryan
Latest posts by Thomas Ryan (see all)
- Intel’s Core i7-8700K: A Review - Oct 5, 2017
- Raijintek’s Thetis Window: A Case Review - Sep 28, 2017
- Intel’s Core i9-7980XE: A Review - Sep 25, 2017
- AMD’s Ryzen Pro and Ryzen Threadripper 1900X Come to Market - Aug 31, 2017
- Intel’s Core i9-7900X: A Review - Aug 24, 2017
