Building the World's 4th Fastest Supercomputer 75
ngkabra writes "In November 2007, a previously unheard of supercomputer called EKA, built by CRL, India came out of nowhere to become the 4th fastest supercomputer in the world. It is also the only supercomputer in the top 10 that hasn't taken any government funding — which means it has no strings attached against commercial exploitation. That is one of the reasons why Yahoo! chose EKA for the cloud computing research that they announced at the Hadoop Summit earlier this week. Yesterday, I attended a presentation by the team that built EKA, and they touched upon a lot of the technical details of EKA, and the challenges faced in designing and building it, which makes for interesting reading."
Interesting maybe (Score:2)
Re: (Score:2)
Re: (Score:1)
Actually I had the same question myself. The talk itself did not go into the details of the interconnect architecture (other than to point out its advantages). I have sent an e-mail to the architect of Eka asking for these details. Should hopefully get some references from him. (In which case I'll post it as an update to TFA.)
In any case, the presentation was targeted towards a general tech audience (i.e. people who don't really have a bac
Re: (Score:2)
Re: (Score:1)
Re:Interesting maybe (Score:5, Funny)
Re: (Score:2)
i'm genuinly interesting in this 'perfect difference' network and would like to understand why it makes a decent interconnect. (btw, i dont think cray has ever built a hypercube machine)
India will be respected (Score:5, Insightful)
Re: (Score:2)
Re:India will be respected (Score:4, Interesting)
Re: (Score:1)
You will be assimilated.
Thought experiment time (Score:4, Interesting)
Let's start with the switches. You can parallelize network traffic - fragment the packets, stick full headers on each fragment, switch the fragments in parallel, reassemble, have the queue re-order inbound traffic by sequence number. (That last step isn't too hard, you have some fraction of the sequence number and the full fragment number map to a unique address in a permanently allocated ring buffer. Copy the payload to that address and the packets are in sequence order, not delivery order.) So, instead of having individual switches that are fast, bank the switches up and have the combined virtual switch run very fast. You can do that on exportable commodity hardware.
Or, you could sneak through Homeland Security (who have much more interest in nipple rings than dangerous weapons) a bunch of 24-channel 5Gb/channel InfiniBand switches. You wouldn't need many of those to get a decent Quake LAN party, and not many more before you could run weapons design software. It is unclear how many are required to run Vista, once service pack 1 is installed.
Interconnects. Obviously, InfiniBand is hellishly fast. So is SCI. 10 Gb ethernet, ideally with iWarp extensions, would be much slower but still perfectly good for a commodity cluster. If you scrap the idea of having machine-to-machine communication and do memory-to-memory communication, you could actually use PCI-e 2.0 as one gigantic interconnect. Ideally, you'd have the memory appear as two separate devices - slave and master - so that direct memory-to-memory RDMA could be initiated. A lot of very similar work has already been done by US supercomputer giants, and given how many have either been bought, gone bankrupt, or otherwise vanished, it's reasonable to suppose large quantities of such RAM could have "migrated" overseas.
CPUs - well, there are some respectable 16-bit pile-of-pc clusters. One was reviewed on Slashdot some time back. Even a cluster of Cell processors, if large enough and well-enough programmed, could be very effective. A hostile nation wouldn't need high-end 4x4 multi-threaded multi-core SMP systems, although again given how juvenile airport security is, I can't imagine it would be hard for someone to export, say, a couple of hundred motherboards at that spec.
Ok, what about OS? Who needs one? Anyone with a copy of OSKit or something similar can work at almost bare metal levels as if they had a full OS. If they did want a full OS, then NetBSD or MOSIX would be quite sufficient. Or they could take an OS project like Exopc and add high-performance networking to it.
Software? If you've a decent copy of BLAS, LAPACK and some solvers, tightly optimized for the platform, you're set. Those core maths functions are critical. Since the functions and API are fixed, it would not be impossible for someone wanting raw power to have put them into an FPGA, SoG or ASIC. Collective operations are also nasty, but they too can be done entirely in hardware, giving you orders of magnitude speedup over conventional software solutions. Synchronizing is the third killer, but there are meta schedulers to handle that and you could again place those on dedicated hardware.
In short, although I couldn't afford to build a top 500 machine, it is only the affording of it that is a problem, and foreign countries are quite well aware of that. Especially after China built its first (publicly-announced) Government-funded Beowulf. Supercomputing is easy, it's the price tag that isn't.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Russia had two major branches in its hardware industry: one based on reverse-engineered clones and a "domestic" one. Neithe
Re: (Score:2)
Re: (Score:1)
Quite a baseball match last night, yes? (Score:5, Funny)
Re: (Score:2)
where's the imagine a beowulf cluster of these post?
I think there would only be need for about six of these in the entire world.
rd
Re: (Score:3, Interesting)
Obligatory... (Score:1)
Finally ....... (Score:5, Funny)
Re: (Score:1)
Re:Finally ....... (Score:5, Funny)
Re: (Score:1)
Sorry (Score:1)
This is old news (Score:4, Informative)
What most people don't know is that this project is funded by the Tata Group, and is directly influenced by TCS (Tata Consultancy Services), which I am strongly related to. The initial goal of this project was to surpass IBM's Blue Gene in the field of supercomputing, but then, things went wrong (as is usually the case with Indian projects), and it landed up in the 4th position, which I still feel, is quite a big feat for a projcet funded privately.
Re: (Score:2, Interesting)
Supercomputer price/performance: Do the math! (Score:2)
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Nice. We like big tatas.
We like nano [slashdot.org] tatas too.
Re: (Score:1)
You mean this tata group? http://www.savethetatas.com/ [savethetatas.com]
Re: (Score:1)
Hmmm (Score:1)
Probably means no...
Top 500 in November 2007? (Score:1)
http://www.green500.org/lists/2008/02/ranks1-100.php [green500.org]
Re: (Score:1)
http://www.green500.org/lists/2008/02/ranks1-100.php [green500.org]
Blue Gene has government funding? (Score:2)
Re:Blue Gene has government funding? (Score:4, Informative)
Re: (Score:1)
If you believe Tata bought this cluster with a business plan to recover its cost in X years, you are mistaken. This machine is (advertisement + ego pumping +
Yeah but does it run ... (Score:1, Redundant)
I'll just have be content with a beowolf cluster of these things imagining me in Russian.
Odd storage requirements... (Score:3, Interesting)
# 80TB storage. Simple SATA disks. 5.2Gbps throughput.
So using the 80TB * 1024GB/TB / 1800blades gives us about 45GB/blade. If they're using "simple SATA disks" this would imply internal disks and 1800 internal disks would have an aggregate throughput much higher than 5.2Gb/s (5.2Gb / 1800 = 2.95Mb/s per disk). Now typically you'd boot the nodes from the network (so you can change the identity of the node easily by booting it from a different image) from some sort of FC array accessed via an IB to FC gateway. However, 5.2Gbps is an odd number to get to since FC comes in 2 and 4Gb formats (1Gb fibrechannel is outdated and 2Gb is on the way out).
While I always see all the CPU details in these articles, I really wish they'd publish more about the storage requirements and methods rather than just staying (we have xTB of disk...). How do they back the finished datasets all up? Tape? MAID? VTL?
Re: (Score:1)
Your comments contradictory
First You say we Indians live in huts, which means we are in stone age but then
you say USA shall bomb us back to stone age!!
You know soon USA Gov. will pay to run simulation on this Cow dung powered computer !!
4th Fastest (Score:2)
The posting title reminds me of the Simpsons gag of Homer imagining what it would be like to be a moderately rich man. "Tee hee! I could rent anything I want!"
Like an old episode of Red Dwarf. (Score:1)
Re: (Score:3, Interesting)
True story: a few months ago, my program was running slowly, and I needed a lot more CPUs. Yelled to the guy a few desks away, and he gave me the names of a few idle compute farms. Job was 50% complete after 10 minutes, I thanked him, and laughingly said that I like to measure my compute power in megawatts rather than MIPS.
Now, a couple of metawatts of compute don't
Fourth fastest according to who? (Score:2)
So then how can this cluster claim to be fourth fastest, when they really are not competing with the publicly funded clusters of the main list?
Obviously, we cannot expect the big corporate supercomputers to spill their beans to contest this, but it
Re: (Score:2)
Funding has nothing to do with it.
Its the useage: As a supercomputer.
20k 1U servers running as shared hosts at an ISP would be a supercomputer if they had the interconnection and the software background. But they arent, so they dont. Same for google, ect.
You will notice a lot of company computers (at airlines, automotive companies, ect, in the top500)
Re: (Score:2)
Dude the site you provided ... was very useful ... I just had to look up the list for November 2007 & look up the details of the fourth supercomputer to check if the original post got it right.
And guess what? Its right there ... don't believe me? Well look up the top 10 [top500.org] list for Nov '07.
And to quote from that site you pointed to:
The TOP500 table shows the 500 most powerful commercially available computer systems known to us.
Bold by me.
Try as I might, I could find no information that this list is only for supercomputers that are publicly funded. May be I didn't try hard enough ... could you
Re: (Score:2)
You WILL find a Microsoft's Ranier cluster running Windows on 2048 processors as the 116th fastest computer in the world on the Top 500 list.
Projective Geometry interconnect ? (Score:1)
This machine has no new concepts or new hardware, boring.
HOWTO (Score:2)
Build the 4th fastest supercomputer:
Want to drop off the end of the Top 500 [top500.org]list? Load SP1.