Supercomputing Upgrade Produces High-Resolution Storm Forecasts 77
dcblogs writes A supercomputer upgrade is paying off for the U.S. National Weather Service, with new high-resolution models that will offer better insight into severe weather. This improvement in modeling detail is a result of a supercomputer upgrade. The National Oceanic and Atmospheric Administration, which runs the weather service, put into production two new IBM supercomputers, each 213 teraflops, running Linux on Intel processors. These systems replaced 74-teraflop, four-year old systems. More computing power means systems can run more mathematics, and increase the resolution or detail on the maps from 8 miles to 2 miles.
Woo! (Score:3, Funny)
Now they can be wrong in hi-def!
Re: (Score:1)
Yep. Greater precision, same "accuracy".
Re: (Score:1)
Yep. Greater precision, same "accuracy".
I was wondering how they ended up with a 74-teraflop -- I guess that explains it. Would have been better with a terraflop though.
Re: (Score:2)
~~
Re: (Score:2)
wooosh [thefreedictionary.com]
Have the solutions converged? (Score:5, Interesting)
I was at a supercomputing conference back in the 90's. There were wonderful reports on doubling the resolution of the grid and so on. Advances in the scale are all good.
The questions are
a) with the increase in detail of the simulations have we converged on a solution. That is do solutions at scale N and 10N match. If they do then the resolution and model are aligned for accuracy in the solution.
b) do the simulations agree with reality.
If a) and not b) then there is something wrong with the model that is not related to compute power or problem resolution, and no amount of compute power will fix it.
Re: (Score:3, Informative)
a) we don't care if the models match
1) the most impotant thing, the one true thing, is that the model remain stable
2) if the model remains stable, then it's judged against reality, not other models. However,
3) Don't break the stability of the model
b) this is a global differential equation. We don't know the initial conditions, and the model can only approximate them. However, the finer the mesh, the closer we can get to the initial conditions, so the further out
Re:Have the solutions converged? (Score:5, Interesting)
TFA ironically begins with the quote '"I don’t think we will ever have enough [computing power] to satisfy us,” says researcher.'
The summary is vague, and the article not much better, and neither say anything about whether the 'new model' is matching observations any better than the old.
It would be nice if they could at least clarify if the sole pair of comparison images are even the same forecast, because the new model shows not only more detail but a completely different prediction.
Come on kids, this isn't a network news sound bite. This is the Internet, and you're a tech news site. Would it kill you to go past the press release?
Maybe I'm just bitter about this because I live in mountains where a coin is a more accurate forecasting tool than the weather service.
Re:Have the solutions converged? (Score:4, Insightful)
Well the possibility is that at some level the weather will exhibit chaotic behavior and no matter how big your model gets it won't matter - the coin flip will be just as good.
Re: (Score:1)
Re: (Score:2)
A science denier would just make an ad hominem attack, like you did.
Re: (Score:2)
Sorry about that.
Re:Have the solutions converged? (Score:5, Informative)
TFL in TFA goes over it.
www.computerworld.com/article/2484337/computer-hardware/noaa-goes--live--with-new-weather-supercomputers.html
It's been a complicated process to get to this point. The NWS has had to ensure that the software running on the new system is producing scientifically correct results. It had been running the old and new systems in parallel for months, and comparing the output.
This comparative testing involved examining output data to determine whether it is numerically reproducible out to five decimal places. There is also a statistical analysis of weather predictions on the new system against the actual weather conditions.
The process wasn't just an examination of numerical data. NWS scientists also studied the weather products and examined them for subtle differences. "There is a lot of human, highly experienced, subjective evaluation," said Kyger.
There are computational differences involved in switching to new chips and a new operating system. They are subtle, and appear in decimal places six through 12.
As you go further out in a forecast, the differences compound. The changes may appear in the fifth day of an extended, five-day forecast as a difference of one degree.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Not surprisingly the scientists that work on weather models care very much about their accuracy. The GFS model's peformance is constantly reviewed: http://www.emc.ncep.noaa.gov/G... [noaa.gov]
Re:Have the solutions converged? (Score:4, Informative)
These simulations are forecasts. They check every forecast against observations, and have very good metrics on how good their forecasts are, and how much skill changes.
See for example how the European ECMWF does its forecasts:
http://www.ecmwf.int/en/foreca... [ecmwf.int]
Every change to the operational model(s) can be and is checked out first against " will it improve the forecast". Similarly improvements in computing power: we simply run yesterdays forecast at higher resolution for example; we can then say "this new model is n% better, but takes 10x as long to calculate", and use that to decide whether its worth buying a faster computer.
On the climate timescale we have a challenge verifying the simulations, but on the weather timescale its straightforward, and done.
Re: (Score:1)
Why is the European Weather Forecast more accurate than the American/USA Weather Forecast model? Also, Intel takes short cuts with their CPU. Intel CPU truncates the math and maybe taking other shortcuts.
You really are pretty clueless about this computing stuff.
Please stop posting about things you only have a vague understanding of. It's not a good look.
And as the resolution increases ... (Score:5, Interesting)
Ultimately the ground features need to be modelled in greater detail to match the increased resolution of the grid.
Which comes own to knowing where each tree/building and similar sized static feature is and how this affects the model.
However, as the grid increases it should not need to know where the butterflies are .
WRF has gotten pretty good, actually (Score:4, Informative)
I'm a computer engineer, not a meteorologist, but I've worked with them off and on for about eight years now. One of the most common models for research use is "Weather Research and Forecasting Model" (WRF, pronounced like the dude from ST:TNG). There are several versions in use, so caveats are in order, but in general WRF can produce really good results at a 1.6KM grid for 48 hours in the future. I was given the impression that coarser grids are the route to happiness for longer period forecasts.
WRF will accept about as much or as little of an initializer as you want to give it. Between NEXRAD radar observations, ground met stations all over the place, two hundred or so balloon launches per day, satellite water vapor estimates, and a cooperative agreement with airlines to download in-flight met conditions (after landing, natch), there's gobs of data available.
The National Weather Service wants to run new models side-by-side with older models and then back check the daylights out of them, so we can expect the regular forecast products to improve dramatically over the next (very) few years.
WRF has gotten pretty good, actually (Score:3)
Re: (Score:2)
Which comes own to knowing where each tree/building and similar sized static feature is and how this affects the model.
Yes, ideally you'd have that much information. But you're still going to gain benefit until your resolution matches your resolution of ground scanning. Right now they've just gotten to two-mile resolution. We have much better maps than that already.
Re: (Score:1)
There are urban airshed models that do exactly this for air quality studies and plume analysis models for hazmat, but I'm not aware of weather forecasting at the block-by-block level. Right off the cuff, I would suspect that albedo is at least as important - at human building scales, reynolds number is going to be pretty high. At that point, it looks more like computational fluid dynamics and less like weather - hence airshed modeling and plume analysis.
Re: (Score:2)
I think we're a LONG way from needed to know information the "down to the tree/building level", but I would be very surprised if they aren't using reflectivity and intensity characteristics from satellite photography (usually about 3 meter resolution). I would think that kind of information would tell them the heat generation/absorption/water evaporation of a particular area reasonably well, they may even have to simplify the data so they can compute it "real time". I work with mapping quite a bit and I c
Merging two topics (Score:1)
HRRR going into production is not directly a result of the supercomputer upgrade. It is a new model that has been in testing for two years, but has just recently been officially "released". While the supercomputer upgrade is great for the increased runs and spatial resolution, it is not directly related to the HRRR being a better model. It will, however, be much more useful when the GFS model is upgraded later this year to significantly increased horizontal and vertical resolutions.
Re: (Score:2)
HRRR means they just run WRF very frequently (http://ruc.noaa.gov/hrrr/). I think it's to help with short range forecasts specifically. As far as I know, they still use GFS primarily (http://www.emc.ncep.noaa.gov/index.php?branch=GFS).
200+ Teraflops (Score:1)
Contributing to global warming, no doubt.
Thanks guys for buying the iSuper. 4x more powerful, 4x the resolution. Come back in 4 years for the retina resolution.
Re: (Score:1)
You're an asshole. Your were born an asshole and you'll die an asshole. And the sooner you die the better off the rest of the world will be.
Should have built a peta-scale system by now.. (Score:1)
Well, congrats to NOAA for the milestone...but isn't a peta-scale system a thing of the past for something as computationally intensive and critical as weather prediction? The prediction model seems to have no scalability issues: "The Hrrr model produces output from the model every 15 minutes versus the previous hourly rate,..", meaning 4x speedup for a 4x upgrade in peak performance. Why didn't NOAA go for a peta-scale system straightaway?
The upgraded systems are already placed way down the list of world's
Re: (Score:1)
NOAA is never particularly well funded and these are not systems that you can just build and immediately throw into production. These "new" boxes are 2 years old and have been operating continuously during that time in an extremely rigorous testing cycle. For all we know the peta scale system is being constructed now and won't be in production til the end of a similar test run.
Re: (Score:2)
While weather forecasts are important, they are not _that_ important. There is no real motivation to be at the top of the pop in computing power for something 'as simple' as a weather forecast.
For some reason people are still stuck in their mind with the "weather forecasts" are 50% luck and 50% wrong myth of the time 30 years ago.
Weather forecasts are actually extremely accurate over a 24h period and very reliable 48h to 72h in future.
Ask anyone who is sailing and checks weather reports more or less daily (
Outsource it (Score:2)
China has a much bigger computer with over 3 million cores.
Also, aren't the two "new" supercomputers 2 years old now?
Re: (Score:2)
Well, what do you expect when the story is a dupe from 14+ months ago? [slashdot.org]
It May help (Score:2)
Re: (Score:2)
Winds that strong are usually only measurable with radar. :)
If there is no weather radar close, no one measures it, obviously. So I doubt there is a 'conspiracy'
However with the 'rise' of the wind industry it would be very important to get a good idea how strong those hurricanes can be in peak gusts.
Think of the weathermen! (Score:2)
This thing is going to put all of the local weathermen out of jobs. I mean how can they predict the weather better than this thing? ;-)
Re: (Score:2)
The local weatherman, if he/she is a meteorologist, looks at several commercial models, the NWS models and adjusts due to his professional experience in an area, he/she is unlikely to go anywhere hell Sonny Eliot (Sonny Idiot) [freep.com] was on the air for 63 years.
Re: (Score:2)
Where are you that weathermen are reliable enough for you to want to keep them around. Around me there's a running joke that you'd be more accurate at predicting the weather throwing a darts blindfolded at a board with all of the seasonal weather possibilities.
Re: (Score:2)
Pixar is going to put the local weathergirls out of their jobs... they already act in front of bluescreens - why not just render them to look and act as desired?
Re: (Score:2)
The uncanny valley.
Nobody wants weather girls that look like trannys.
this article is a year old (Score:2)
July 2013.
Re: (Score:3)
Re: (Score:2)
...discusses transitioning from an old model that used a 27 kilometer resolution to a new model...
what of data resolution? (Score:2)
Has the resolution and reliability of initial data points improved as well? Or are we just doing a finer interpolation of model output with same data input?
what of data resolution? (Score:1)
There have been advances in assimilating radar and satellite data into models, which can provide better resolution than data from other observation platforms such as profilers, raobs, and surface observing stations. This reduces the time needed for models to spin up.
The real advantage from improving the resolution is the ability to explicitly represent processes in a way that couldn't be done with coarser models. This is especially true in regard to convective processes. A thunderstorm updraft might be a fe
Mod parent up (Score:1)
I posted, so can someone else mod the parent up? tnx.
Re: (Score:2)
That has to be perhaps the best AC post I've read on /. in a very long time, perhaps ever. Well done.
Here's what the code looks like (Score:2, Funny)
while (1) {
prediction = PredictWeather();
if (prediction == true) {
AskForMoreGrantFunding();
BlaimGlobalWarming();
} else {
BlaimGlobalWarming();
AskForMoreGrantFunding();
}
}
Re: (Score:1)
"Blaim"? AskForMoreEducationFunding();
Not the whole story - maybe leaked as a CMA action (Score:3)
This was an interesting article. However as UW's Cliff Mass has previously pointed out [blogspot.com] (And, today, he privately confirmed is still the case), NOAA is sitting on already-approved funds to purchase a weather modeling computer that's seen as a potential "game changer" for US climate modeling.
Over a year ago Congress approved the purchase of a computer that's roughly an order of magnitude more powerful than the pair mentioned in this article - but, because NOAA has a contract with IBM and IBM recently sold their server business to Lenovo, NOAA has been sitting on their hands regarding approval of the purchase of such a computer from a Chinese company.
So while the improvements mentioned in the article are better than nothing... in truth we should be a significant step beyond that by now.