Kioxia and Dell Cram Nearly 10PB Into a Single 2U Server (nerds.xyz) 17
BrianFagioli writes: Kioxia and Dell Technologies say they have built a 2U server configuration capable of scaling to 9.8PB of flash storage, which is the sort of density that would have sounded impossible just a few years ago. The setup combines a Dell PowerEdge R7725xd Server with 40 Kioxia LC9 Series 245.76TB NVMe SSDs and AMD EPYC processors. According to Kioxia, matching the same capacity with more common 30.72TB SSDs would require seven additional servers and another 280 drives.
The companies are pitching the hardware squarely at AI and hyperscale workloads, where storage is rapidly becoming a bottleneck alongside compute. Kioxia claims the denser configuration can dramatically reduce power consumption and rack space requirements while remaining air cooled. The announcement also highlights how quickly enterprise storage capacities are escalating as organizations race to support larger AI models, massive datasets, and increasingly demanding data pipelines.
The companies are pitching the hardware squarely at AI and hyperscale workloads, where storage is rapidly becoming a bottleneck alongside compute. Kioxia claims the denser configuration can dramatically reduce power consumption and rack space requirements while remaining air cooled. The announcement also highlights how quickly enterprise storage capacities are escalating as organizations race to support larger AI models, massive datasets, and increasingly demanding data pipelines.
No object (Score:5, Insightful)
>"The companies are pitching the hardware squarely at AI and hyperscale workloads, where storage is rapidly becoming a bottleneck alongside compute."
And where, apparently, price is no object. I wish they would focus on that crap and leave normal business and consumer-sized parts alone so we can afford them again.
>"Kioxia claims the denser configuration can dramatically reduce power consumption"
So the AI datacenters can just buy more of them in the same space and still strain all the grids as much so consumer electricity prices continue to rise.
>"The announcement also highlights how quickly enterprise storage capacities are escalating"
While consumer-grade storage capacities are stagnant or even REDUCING just so people can get by.
I wish this bubble would burst sooner than later.
Power Requirements (Score:5, Insightful)
Re: (Score:2)
It doesn't really do you much good to fit it into 2U if you need 20-30kW for that one box.
Re: (Score:2)
Re: (Score:2)
Something tells me that if you can afford the $48,000 price tag for just one of 40 245TB drives in that insanely dense 2U server, you probably also have the money to build your own power generation.
Re: Power Requirements (Score:2)
Re: (Score:2)
It doesn't really do you much good to fit it into 2U if you need 20-30kW for that one box.
I saw some statements from Nvidia about how they want 1 MW racks. This would fit. At that power density you need liquid cooling, which I don't think this box has.
To get a 1 MW rack you need really interesting power distribution too. I think they wanted to go to 800 V DC to each rack. Care on the part of the maintenance techs is encouraged.
40 NVME ? (Score:3)
How does 40 NVMEs fit in one PCIe bus?
Or are they connected over another interface thar is slower, then into PCIe.
Can someone knowledgeable answer this?
Re: (Score:3)
Dual socket epyc can have 160 lanes.
Coincidentally, 40 nvme drives at x4 would use 160 lanes.
However, that would be useless, since you need connectivity beyond the box and misc needs for pcie. So either x2 per drive or pcie switches.
Re: (Score:3)
How does 40 NVMEs fit in one PCIe bus?
Via a PCIe switch backplane. They've been around for a while... Perhaps as far back as 2012...
I fully expect SAS4 to be the end... NVMe-OF will replace SAS, and the drives will plug into the crazy 800GbE switches that are available now. Not on the drawing board... Now.
T
Re: (Score:2)
Yes, though I don't know about nvmeof. I feel like san style block is overall less popular than other sorts of software approaches to distributed storage nowadays.
Storage people keep pushing the way it was done with fiber channel attached controllers abstracting things to generic block devices. Shared sas, fcoe, iscsi/iser... Have seen so many tries at bringing the concept and being ignored in favor of things like clustered filesystems and object store.
Just like hardware raid controllers are nearly non e
Re: (Score:2)
Storage people keep pushing the way it was done with fiber channel attached controllers abstracting things to generic block devices. Shared sas, fcoe, iscsi/iser... Have seen so many tries at bringing the concept and being ignored in favor of things like clustered filesystems and object store.
Clustered FS and Objectstore are built on top of SAS, FCOE, iSCSI, NVMe-OF. You first have to solve the problem of packing thousands of storage devices within the signal integrity radius of the transport medium before you can start abstracting. For NVMe that radius is about 1.5 - 2 meters from the CPU socket. SAS about 5 meters. Not sure on FC, I presume a couple km.
Just like hardware raid controllers are nearly non existent in nvme world
Completely common. Like 70% of all servers sold include a RAID controller that can talk to NVMe devices. But there's a catch... They suck
I think I have dyslexia ... (Score:2)
Kioxia and Dell Cram Nearly 10PB Into a Single 2U Server
Re: (Score:2)
>" ... 'cause now I'm craving peanut butter and jam sandwiches."
Well, you can certainly buy a hell of a lot of PBJ sandwiches for just one of those SSD's. Probably way over a lifetime's worth for one of those servers.
Re: (Score:2)
Density AND speed (Score:4, Informative)