The short story is that you can now take a SeaMicro SM10000 chassis, pull out one of the 12 core Atom cards, and plug a Xeon card in. Instead of 12 wimpy cores with 24GB of memory, you get 4 beefy 2.4GHz Xeon iSomethingmeaningless cores with 32GB of DDR3. To make things better, when you go from 768 cores per 10U chassis to 256 bigger ones, you can do it one card at a time, plug and play, mix and match.
The BBOSNS (Big Box Of Shared Nothing Servers) is now heterogeneous for CPU types. Best of all, it works in the same power envelope, 3.5Kw/chassis, with either card, and you can hot swap them. Our lawyers insist that we mention using adequate eye protection when doing so for no particular reason we will admit to in print. It looks like this.
First generation 8 Atom board on the left, new Xeon board on the right
The magic is once again in the SeaMicro ASIC code named ‘Freedom’. There weren’t many details on the part, but since SeaMicro is claiming 10Gbps I/O for each socket, so that would mean two PCIe2 lanes. Freedom is most likely just an older SeaMicro ASIC with the external channels bonded to one link, and some logic added to smooth that over, but there could be more lurking under the hood.
Samsung was brought on board for their memory, something the company does extremely well. Remember those mystery DDR3 ECC SoDIMMs that we were confused about at IDF? Well, they aren’t actually for Atoms, they are for, wait for it, the Xeons that slot in to the place where Atoms used to be. The memory uses 30nm class 4Gb chips and runs at 1.35v, basically tuned for low power use. This is what Samsung is calling ‘Green’ memory, and is not using the 20nm 1.25v parts they were also showing in September.
SeaMicro picked the Samsung memory not because Samsung has a cool corporate logo, but because it is the lowest power memory you can buy with any regularity. If you run the numbers on cores, cards, and the rest, you will realise that a single SM10000 has 64 card slots, each of which can take 4 SoDIMMs, 2 * 2 channels, of DDR3/1333. If you use 8GB ECC SoDIMMs, that would mean 2TB of DRAM per 10U chassis, or a little over 200GB and 6 sockets per U.
Samsung is claiming that the power used by these sticks is a mere .29W/GB vs ‘normal’ DDR3 pulling about 1W/GB. Multiply that by 2048GB/chassis, and the power savings are a staggering 1450W or so. Even if Samsung is exaggerating their performance and giving a worst case for the competition, the savings are still massive. Instead of using over half the chassis’ power budget on memory, you use about a sixth. Not bad.
Getting back to power, we come to Intel. The Xeons in question are not normal Xeons, these parts are E3-1260L, parts you probably won’t find in the insert Intel puts in this Sunday’s newspaper. These are special parts, 45W 2.4GHz 4-core Bromolow chips, picked, fused, and painted especially for the spring SeaMicro server and fashion extravaganza. Why? Low power. They are at the most efficient part of the performance per watt curve, and should put most anything else to shame in that metric. If you read the spec sheet, you will see this is one very odd chip, can you think of any other iSomethingmeaningless or Xeon with only 2 PCIe2 lanes?
More importantly, we come to what SeaMicro did to these already odd chips. The box stuffer has named this little bit of silicon tweaking TIO or Turn It Off. If you recall, SeaMicro has some very unique capabilities built in to their servers, namely faking all I/O, storage and everything else with their ASIC, and exposing virtual interfaces to the CPU. The entire chassis has a number of I/O ports, storage, and management engines, but they are all virtual and shared.
The storage can be allocated arbitrarily by socket, including mapping all sockets to a single disk, or the other way around. You can mix and match magnetic, SSD, and anything else SATA based that will fit in the 64 drive bays. You can even mount a drive read-only and map every socket in the system to it as a PXI boot volume if you want. You have a virtual and transparent SAN in a box with each chassis.
This means the stuff on the Xeon chipset, Ethernet links, 18 more PCIe2 links, SATA controllers, and everything else, sit unused and unusable. Same for the GPU on the Xeon, and likely some other bits. They aren’t needed, so TIO at as low a level as possible, and keep it there. SeaMicro doesn’t even run power to leads to unnecessary bits, where possible, for lower power. The savings here are probably not massive compared to C6 power gating, but once again, SeaMicro doesn’t seem to leave any stone unturned on the wattage front.
In the end, the claim is pretty impressive, 1024 ‘real’ cores, 8TB of DRAM, and a few bits of open space per 42U rack. Performance per U went up by about 20%, and bandwidth per socket went up by 4x too. Lots of ‘small’ core chips can beat this density, and several blades can exceed the ‘real core’ CPU density, but we don’t think anyone can come close to the performance per watt capabilities of this box when fully outfitted. It may be shared nothing architecture, but it is flexible, available in volume now, and doesn’t have any caveats attached. Not bad, not bad at all.S|A
Latest posts by Charlie Demerjian (see all)
- AMD’s Epyc has lots of connections - Jun 26, 2017
- AMD’s Epyc is a major advance in security - Jun 22, 2017
- Intel talks about Skylake’s mesh interconnect - Jun 15, 2017
- Intel announces X-Series and more without details - May 30, 2017
- AMD talks Threadripper, Ryzen Mobile, and Ryzen Pro - May 22, 2017