Can Anyone Suggest a Good Switch? 54
wgadmin asks: "I am a sysadmin for a 500-node Linux cluster. We are doubling the size of our compute nodes, so, as a consequence, we need a new switch. We currently have a Foundry FastIron 1500 -- however, a) it doesn't have enough ports (only 208) and b) it is extremely unreliable. We want something that's solid as a rock. We process mostly serial jobs. And we will probably require ~320 ports. What's everyone using for their HPC clusters? There's so many high performance switches on the market, we hardly know where to start."
How much better than Cisco? (Score:3, Interesting)
Cisco is the de-facto brand of networking gear for standard stuff like Ethernet. How much better are these high-performance switches people are talking about as suggestions in the comments here? This is not a rhetorical question, I just realy want to know and I'm too lazy and uninterested to look into it myself, but not lazy enough to stop typing this slashdot post. Is it enough to be worth going with non-Cisco for HPC clusters that use Ethernet-based interconnects? I know Cisco isn't infallible, but for all kinds of reasons they're a good bet in networking gear come purchasing time, at least outside this HPC cluster business.
Re:Several things left out. (Score:2, Interesting)
One of our sites tried them.... and practically kicked them straight out of the window. If you care about throughput then you want to stay well clear.
Re:Forget ethernet (Score:3, Interesting)
Re:Extreme Networks (Score:3, Interesting)
We have two Linux clusters -- Viking, a 512 processor P4 cluster, and Mars, a 400 processor Opteron cluster (being commissioned). We also host a number of large Sun machines, including SunSite Northern Europe.
The department also hosts a 250+ node teaching lab and several floors of staff and research desktops, each with 100Mbit+ to the desk and a 1Gbit uplink from each switch. At the middle of our network are two Black Diamonds, one of which is the new 10ks which we helped Extreme beta-test (now in production). A third BD is being set up at a second site with 10Gbit fibre links.
The rest of the network ('the edge') is also made up of Extreme switches, ranging from Summit 1s, 24s, 48s to some of their newer 400-series [extremenetworks.com] 1u switches on Mars and elsewhere. (The 400-series can run 10Gbit fibre uplinks, and each copper port runs at up to 1Gbit, which is ideal for a cluster environment.)
Plus, they're are all coloured a nice shade of purple.
build a fat tree (Score:3, Interesting)
tree with 12 roots for $8000. spend more to get
more cross-section bandwidth, less for less. it
scales with your budget.
Re:Cisco 65XX (Score:2, Interesting)