Beowulf In Business 46
"Cnet has a story on how businesses are starting to use Beowulf for those heavy duty tasks," writes NIB. The story notes, "Beowulf wouldn't be good for a program that executes large numbers of transactions, such as an airline
reservation system. It would, however, work for business tasks such as deciding how to design an assembly line
or which mix of currencies to buy, said International Data Corporation analyst Dan Kusnetzky."
MOSIX.. (Score:1)
Re:Real-world applications for clusters (Score:1)
Re:Clustering (Score:1)
This was to be an ad hoc "Stone Soupercomputer" style configuration built out of machines brought to the conference by attendees.
Re:Real-world applications for clusters (Score:2)
One big one is simulations for financial calculations. One such is roughly this: the price of some class of security is sensitive to interest rates. So you want to see what happens if, at several time steps, the interest rate goes up or down some small amount. Evaluating the different 'paths' of interest rates over time lends itself to parallel processing.
Re:PVM and Beowulf (Score:1)
Re:Aren't you clever (Score:1)
Re:Not good for large parallels? (Score:1)
As long as there was not one huge shared database, of course. And to dump old databases to semi-offline storage, a second backbone could be installed in these file servers to push the data onto backup servers, which would merge the databases again, and write them out. A lot of investment in hardware, but much less than a similar proprietary system.
Re:Not good for large parallels? (Score:1)
There's going to be some interesting developments in this area for Linux soon, even if it means I'm going to have to start them myself.
Re:Story sort of missed the boat (Score:1)
Whether or not they used an open-source operating system is not the point. The goal was to provide an MPP system for as little cost as possible. If a collection of Tru64 UNIX workstations operating in a Beowulf cluster provides more computing performance at less cost than a similar system from a major MPP vendor, then it seems to meet the criteria for why Beowulf began. Sure, it might cost more than the same Alpha workstations running Linux, but it is also likely to perform better. Life's little tradeoffs are everywhere, aren't they?
Cheers,
David Hull
david.hull@england.com
Re:Not good for large parallels? (Score:1)
If only there were a transparent VM... (Score:1)
I have set up a few Beowulf machines for S&G. I used PVM [freshmeat.net], RH Linux 6.0/5.2, a 10/100 switch, and about 6 boxen. It worked quite well, except that it took a few days to get operating how I wanted. I wrote a couple applications to crunch numbers across the cluster, tested throuput, etc. For even more S&G I used MP3PVM [freshmeat.net] to RIP a few CD's real fast. Fun!
Now this is all well and good, but wouldn't it be great if we could have a transparent virtual machine that runs across all the nodes? Something which you could use "/bin/bash" on as your command shell.
Now, I am not sure how this would be accomiplshed-- forinstance how you would effciciently share memory accross machines or decide how to break up tasks (break on thread, would be one way); this is just to open up conversation.
Imagine: Lower your SETI@Home WU time to mear seconds :) (is it far to run a distributed computer under a distributed computer?)
-AP
Not good for large parallels? (Score:1)
Re:Not good for large parallels? (Score:2)
The largest airline systems all use IBM's Transaction Processing Facility (TPF), which is a specialized real-time OS for mainframes. TPF shares disks amongst all processors in the cluster and pushes the locking down to the individual disk controllers (specialized microcode). Where I work, we get about 200,000 physical I/Os per second to the disk farm, using TPF on eight mainframes and a several TB of disk.
Still, there's nothing about large-volume OLTP that Linux couldn't do, it's just a matter of programming. I, for one, would like to see it happen.
database locking (Score:1)
of course, with gigabit ethernet...
"Subtle mind control? Why do all these HTML buttons say 'Submit' ?"
Clustering (Score:2)
There's a movement on to put together a large Beowulf cluster for the Boston Geekfest in October. One of the things we're trying to come up with is a good demo that actually shows something to the crowd. We've had ideas from the realtime rendering of POV scenes to decryption (yeah, right, watch it hum for 20 hours and then spit out the true key) but haven't come up with a "killer demo app". Email if you have any ideas.
Re:database locking (Score:1)
doesnt do much for the latency. Most of the latency is in software (TCP/IP etc) that shared memory systems avoid.
Real-world applications for clusters (Score:1)
Re:Not good for large parallels? (Score:1)
Re:database locking (Score:1)
Re:Not good for large parallels? (Score:1)
distibuted DB, like the one you need in this kind of environment... but a distributed FS may solve the probem:))
This would rock (Score:1)
heh... If Only...
Re:oh yeah (Score:1)
Story sort of missed the boat (Score:2)
I just read the article. As a manufacturer of "turn-key" Beowulf systems, here was my reply to the author:
Stephen,
I just read your story about Beowulf systems. While the story was well written and informative, there are some points that you have missed.
1) The definition of a Beowulf requires and "open source" OS (See "How to Build a Beowulf" by Sterling, Becker, Salmon, Savarese) Therefore, systems built from True 64 are NOT Beowulf systems.
2) You missed my company, Paralogic Inc. We sell turnkey Beowulf systems. In fact rather than "several" as reported by IBM, we have several dozens of installed production systems at companies like Lucent, Amerada Hess, Conoco, Procter and Gamble, government sites like NASA, NRL, and the Air Force, and many Universities. (see www.xtreme-machines.com [xtreme-machines.com])
3) There is a rather huge barrier to entry because of the technical nature of these machines. As far as I know, we are the only company who will offer support for Beowulf clusters. Without support the market can never enter the mainstream.
4) There have been quite a few other people who contributed quite a lot of effort to the Beowulf technology other than IBM and VA Linux. Although all contributions are welcome, these guys are a little late to the party and we hope they stay.
Sincerely
Douglas Eadline, Ph.D.
President
Paralogic, Inc. [plogic.com]
PEAK PARALLEL PERFORMANCE
Re:oh yeah (Score:1)
Re:Real-world applications for clusters (Score:1)
3 p120s (Score:1)
Other than learning a new technology, i don't have a real use for a parallel processing machine, i just do basic php -> mysql stuff with small dbs...
however, it sounds like a really cool thing to set up, and i want to learn. Alot of these sites talk about beowulf alot, but don't give an explanation on how to set one up! Either i'm looking at the wrong sites or i just don't kow how to do it..
Can anyone point me in the right direction or give me some tips on doing this? And responces like *just give me those three pcs* are appreciated but will be ignored
Thanks for the help
jc
--yep. i'm a NEWBIE!!!
Re:3 p120s (Score:1)
How to Build A Beowulf : A Guide to the Implementation and Application of PC Clusters;
Sterling, Thomas L. / Becker, Donald J. / et al.
http://www1.fatbrain.com/asp/bookinfo/bookinfo.
High Performance Cluster Computing : Architectures and Systems, Volume I; Buyya, Rajkumar
http://www1.fatbrain.com/asp/bookinfo/bookinfo.
High Performance Cluster Computing : Volume 2, Programming and Applications
http://www1.fatbrain.com/asp/bookinfo/bookinfo.
What beowulf is and isn't good for. (Score:1)
This is a relatively large domain or problems, but it doesn't work for everything. A lot of business applications require high reliability and availability. If you use beowulf, you have to implement these features for your application on your own.
The simulations that businesses are running on these things aren't really in the same league. For the most part, they aren't time critical and if a failure occurs that invalidates a test run, they can ususally be rolled back to some midpoint and started again without a significant loss of time.
Beowulf isn't just useful for CPU intensive tasks though. All those processors also provide significant amounts of memory bandwidth and all those machines provide potentially large amounts of disk storage and bandwidth, but again, you need memory or disk intensive tasks that can easily be split out to many loosely coupled nodes.
Re:3 p120s (Score:2)
Re:database locking (Score:1)
Re:Clustering (Score:1)
The problem is that there will probably not be anyone there that could even beat a single computer |).
LINUX stands for: Linux Inux Nux Ux X
Mfg Design? Depends on your tool... (Score:1)
OTOH, if you're doing linear or integer programming, those are parallelizable if you're doing branch & bound stuff. But LP/IP doesn't always give you the granularity you need to make decisions as accurately as DES does (and can't account for the stochastic and dynamic nature of manufacturing processes).