Intel once again dodged even softball Xpoint/Optane questions at their ‘launch’, it kinda makes you wonder. Actually SemiAccurate doesn’t wonder about this one, Intel can’t answer simple questions because it would make the product look really bad.
In case you missed Intel’s triumphant, multi-year late ‘launch’ of Apache Pass aka Xpoint/Optane DIMMs, well you didn’t miss much. SemiAccurate wasn’t invited because we might ask questions and Intel is desperate to avoid such awkward interludes. Once again there was an embarrassing launch with little details and full of lifestyle choice options. Intel has nothing real to actually show off but they can’t keep from talking about it. First rule of holes guys…
We will spare you the long version of the ‘launch’, we put that in quotes because nothing was actually launched yesterday. Instead there was a lot of talk, a few vague specs like 128, 256, and 512GB capacities, but that’s it. Why is that not a launch? “The modules are pin-compatible with standard DDR4 DIMMs and will be supported by the next generation of Intel’s Xeon server platforms.” Do note they mean Cascade Lake which isn’t out.
So yesterday there was a dog and pony show for things that Intel wants you to talk about and extol the virtues of, but they won’t actually talk about. SemiAccurate hears Intel is wondering why they aren’t taken seriously by anyone anymore, but we can’t explain it, *COUGH* really *COUGH*. The most farcical part of yesterday’s ‘event’ was the Q&A session at the end where Intel had 11 questions, all softball ones, asked of them. They dodged all but two, one of which was about a code name, even though some of the questions were previously answered by the company.
You can read the questions at Anandtech’s live blog of the event, but be forewarned it is pretty painful and content free. This is by design, not a slight on the reporting skills of the writers. Anandtech has the patience of saints. Luckily for readers, SemiAccurate can answer most of these questions and you will understand why Intel didn’t answer them. Better yet we will tell you a lot more interesting things Intel REALLY doesn’t want you to know about Xpoint/Optane/Apache Pass. (Note: Questions and Intel answers copied from Anandtech, our commentary after the S|A -)
1) Q) Do the DIMMs require next gen CPUs? A) Yes
S|A – You need Cascade Lake because it was too buggy in Purley. The hardware support was there as far back as Sandy Bridge-EP/Romley as we told you about exclusively in 2012.
2) Q) You mentioned Intel will be revenue shipping for 2018. Will ecosystem partners be shipping systems in 2018? A) We expect systems to be shipping in 2018, but the details of specific customers will be up to them.
S|A – None of our moles say endurance is anywhere near enough so don’t expect to actually be able to buy them. Note Intel didn’t say they were going to be available to the general public, something their messaging on developer programs and “broad availability in 2019” strongly suggest against. Don’t hold your breath here. Also do note they used very similar language on 10nm messaging for a long time now.
3) Q) What is Crystal Ridge? A) Crystal Ridge is the top level family of persistent memory parts, of which Apache Pass is a member
S|A – As we told you in May, 2015.
4) Q) It was mentioned about DDR4 pin compatibility with the DIMMs. Can users mix and match the DIMMs on individual channels, or will there be dedicated channels? A) Not disclosed at this time.
S|A – This one is messy. Originally the plan was for Purley to put a single DRAM DIMM into slot 0 of a memory channel and then the other slots, note the pleural, be either DRAM or XPOINT. 3 slots became 2, and then disaster hit.
Two years ago or so, Intel changed their messaging on Apache Pass to ‘recommend’ that you dedicate channels to Xpoint only. Why? Random write failures. When you write to Xpoint, sometimes it doesn’t work, and Intel can’t figure out why. This isn’t to say the cell is bad, if you retry it, things almost always work fine. Given how slow Xpoint cells are vs DRAM, this ties up the memory bus for enough cycles to destroy memory performance and hobble the bus.
Last we heard these problems had not been solved and the official word was still to not mix and match. We would ask but since Intel won’t officially answer much easier questions, don’t expect honest and forthcoming info here. Don’t be surprised if this functionality never sees the light of day though, from what SemiAccurate’s moles are saying, this is a fundamental flaw with Xpoint that can be papered over by an SSD controller but not on a memory bus.
As a related aside, take a look at the specs of the developer systems as reported by Anandtech. 192GB of DRAM means 64GB on each of the six DRAM channels, so far so good. 1TB of Xpoint means that 1TB/6 channels equals… well not 128, 256, or 512GB. This means the Apache Pass developer systems shipping in 2018, maybe, aren’t actually Apache Pass. Doh! So much for honest messaging.
Once again Intel seems to have ‘omitted’ critical information in a way that could lead people to believe a broken product is in better shape than it really is. There was no need to jump through these hoops just for an inevitable self-inflicted wound, but Intel is not one to waste an opportunity. That said SemiAccurate is not one to tell them how they could do it both honestly and give out a better message even if it isn’t all that complex.
5) Q) One of the slides had a Xeon Platinum logo. Will the DIMMs only work on Xeon Platinum? A) Not disclosed at this time. We just showed pairing the best with the best.
S|A – Last time we looked at the Cascade spec sheets, Apache Pass compatibility was for a subset of Platinum SKUs and a single Gold SKU which may be sold only to large OEMs. In short you get soaked. That said you MUST pick a SKU with the memory surcharge on Purley, $3000, but there are potentially some configs of Cascade Lake that will avoid it.
Do note that those configs don’t make sense for the real world, they are just there as a straw man to deflect criticism.. The method for this skirting is because the breakpoint for the tax goes from 768GB in Purley to 1TB in Cascade Lake. Yay? For the technical, those configs would be 768GB of Xpoint (6x 128GB DIMMs) plus up to 192GB of DRAM.
6) Q) Will there be a DIMM writes per day? A) Not disclosed at this time. But DIMMs will be operational for the lifetime of the DIMM
S|A – If this isn’t a giant waving red flag complete with sirens and flashing lights I don’ t know what is. Endurance is _THE_ critical question and why Xpoint is now many years late. It didn’t work before and still is woefully inadequate. It wasn’t brought up in the presentation and obviously avoided in the Q&A.
The Xpoint SSDs are massively overprovisioned even though Intel flatly denies they are. If you count the memory channels on the SSDs, multiply that by the size of the chips, you get a capacity much higher than listed on the drive. Intel claims this isn’t overprovisioning but flatly refuses to say what it is for, feel free to make up your own mind. That said Intel claimed the next round of Xpoint SSDs raise the limit from 30DWPD to 60DWPD, a meaningful increase but largely irrelevant in SSD form factors. We told you the code name for this a few days ago.
On a DIMM form factor 60DWPD is at least a couple of orders of magnitude less than needed. While an SSD controller can remap sectors and even bytes on the fly, a system memory controller can’t do that without tanking the system latency. There are a lot of programs that can hit the write lifetime of a small block of Xpoint memory in seconds. Think there was a reason Intel avoided this topic like it was radioactive? Endurance is still not there for DIMMs.
7) Q) What is the power draw of the Optane DIMM compared to regular DIMMs? A) Not disclosed at this time.
S|A – Like the mix and match issue, this is another ‘hidden’ bug that Intel is struggling with. Power is way above where Intel claimed it was years ago, and they have been struggling to lessen it for years. These efforts have been partially successful but our moles say things aren’t good enough yet. The biggest problem at the moment is that writes are inconsistent for power draw. Some writes are within the claimed window but a large enough portion is higher, sometimes significantly so, that the overall power budget is blown out. This may or may not be related to the mix and match killing bugs.
8) Q) Does this mean the memory limits of future Xeons are being increased? A) Not disclosed at this time.
S|A – As we said above Cascade will be at 1TB instead of Purley’s 768GB, but don’t expect it to come any cheaper than the $3000 Purley asks. Think value, not extortion.
9) Q) Will all future Xeons support the Optane DIMMs? A) Not disclosed at this time.
S|A – No, only the ones you pay a lot extra for.
10) Q) Clock speeds of the DIMMs? A) Standard DDR4 speeds. Other details not disclosed.
S|A – The memory channel runs at full DDR4 clocks last time we saw details. The Xpoint cells run much slower and so writes tie up the bus for a lot longer than many are comfortable with. This is part of the mix and match delay, but not as crippling as the failed write issue.
11) Q) Will the QLC NAND adhere to JEDEC data retention. A) Not disclosed at this time.
S|A – We will have to end this one with a tease because we are still researching some bits. Lets end with, “There is a reason Micron took a different direction”.S|A