Google's New Weather Prediction System Combines AI With Traditional Physics (technologyreview.com) 56
An anonymous reader quotes a report from MIT Technology Review: Researchers from Google have built a new weather prediction model that combines machine learning with more conventional techniques, potentially yielding accurate forecasts at a fraction of the current cost. The model, called NeuralGCM and described in a paper in Nature today, bridges a divide that's grown among weather prediction experts in the last several years. While new machine-learning techniques that predict weather by learning from years of past data are extremely fast and efficient, they can struggle with long-term predictions. General circulation models, on the other hand, which have dominated weather prediction for the last 50 years, use complex equations to model changes in the atmosphere and give accurate projections, but they are exceedingly slow and expensive to run. Experts are divided on which tool will be most reliable going forward. But the new model from Google instead attempts to combine the two.
"It's not sort of physics versus AI. It's really physics and AI together," says Stephan Hoyer, an AI researcher at Google Research and a coauthor of the paper. The system still uses a conventional model to work out some of the large atmospheric changes required to make a prediction. It then incorporates AI, which tends to do well where those larger models fall flat -- typically for predictions on scales smaller than about 25 kilometers, like those dealing with cloud formations or regional microclimates (San Francisco's fog, for example). "That's where we inject AI very selectively to correct the errors that accumulate on small scales," Hoyer says. The result, the researchers say, is a model that can produce quality predictions faster with less computational power. They say NeuralGCM is as accurate as one-to-15-day forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF), which is a partner organization in the research.
But the real promise of technology like this is not in better weather predictions for your local area, says Aaron Hill, an assistant professor at the School of Meteorology at the University of Oklahoma, who was not involved in this research. Instead, it's in larger-scale climate events that are prohibitively expensive to model with conventional techniques. The possibilities could range from predicting tropical cyclones with more notice to modeling more complex climate changes that are years away. "It's so computationally intensive to simulate the globe over and over again or for long periods of time," Hill says. That means the best climate models are hamstrung by the high costs of computing power, which presents a real bottleneck to research." The researchers said NeuralGCM will be open source and capable of running on less than 5,500 lines of code, compared with the nearly 377,000 lines required for the model from the National Oceanic and Atmospheric Administration (NOAA).
"It's not sort of physics versus AI. It's really physics and AI together," says Stephan Hoyer, an AI researcher at Google Research and a coauthor of the paper. The system still uses a conventional model to work out some of the large atmospheric changes required to make a prediction. It then incorporates AI, which tends to do well where those larger models fall flat -- typically for predictions on scales smaller than about 25 kilometers, like those dealing with cloud formations or regional microclimates (San Francisco's fog, for example). "That's where we inject AI very selectively to correct the errors that accumulate on small scales," Hoyer says. The result, the researchers say, is a model that can produce quality predictions faster with less computational power. They say NeuralGCM is as accurate as one-to-15-day forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF), which is a partner organization in the research.
But the real promise of technology like this is not in better weather predictions for your local area, says Aaron Hill, an assistant professor at the School of Meteorology at the University of Oklahoma, who was not involved in this research. Instead, it's in larger-scale climate events that are prohibitively expensive to model with conventional techniques. The possibilities could range from predicting tropical cyclones with more notice to modeling more complex climate changes that are years away. "It's so computationally intensive to simulate the globe over and over again or for long periods of time," Hill says. That means the best climate models are hamstrung by the high costs of computing power, which presents a real bottleneck to research." The researchers said NeuralGCM will be open source and capable of running on less than 5,500 lines of code, compared with the nearly 377,000 lines required for the model from the National Oceanic and Atmospheric Administration (NOAA).
About damned time (Score:5, Insightful)
Re: (Score:1)
Re: (Score:1)
I look forward to increased accuracy of weather reports
But what you'll actually be getting is "AI" hallucinations about the weather, so don't forget to look out of the window... And up!
Re: (Score:2)
I look forward to increased accuracy of weather reports
But what you'll actually be getting is "AI" hallucinations about the weather, so don't forget to look out of the window... And up!
The real question is whether it will accurately predict your chance of dying from Boeing 737s losing parts which plummet to the Earth as they fly overhead. Enquiring minds want to know!
Re: (Score:3)
Yes. The chance is zero. If it's a Boeing, I ain't going.
Re: (Score:2)
"Going" or not bears no difference if parts fall off the aircraft through your roof onto your head.
Source: the Alaska Airlines flight that lost it's door flew more-or-less right over my house, and that door landed about a mile down the hill in someone's back yard trees.
Re: (Score:2)
Wow, what a strike of luck. Still, while somewhat higher than a hit by a meteorite, I guess the probability of a hit by a falling Boeing wing is still low enough. Amazing, though, that today I can meet you, the guy who actually was under that particular flight path, online.
My CSB is that I was flying with Malaysia airlines to Europe and back quite often for a while back in the 2010s, and wasn't on the MH17 that got shot by the crazy ruzzian only because an unexpected delay kept me in Holland for a few days
"Traditional" Physics? (Score:2)
Re: (Score:2)
"Traditional Physics" is Physics done traditionally without the marketing department.
Marketing dept: If you were solving a large system of coupled diff. equations with a complex set of initial conditions and constraints (data), you MUST now call it "Data science".
Marketing dept: If you were tuning model parameters by brute force Monte-Carlo style simulations, you MUST now call it "training".
Marketing dept: If you were data fitting, now you are now "machine learning". If you were extrapolating, now you are "
Re: (Score:2)
Re: About damned time (Score:2)
Re: (Score:2)
Re: (Score:2)
Your weather predictions might be off because you have an iPhone. I've noticed they seem to be less accurate. YMMV, and obviously it's because of the data sources, not the OS.
Agreed. My company issued iPhone's weather software is only good for temperature and humidity. Predicting the weather is a crapshoot at best.
Re: About damned time (Score:2)
Re: (Score:2)
They don't use their own temperature sensor in the phone hardware, of course. They usually use specific data collection points of the different weather brokers. A suburb may be getting its temperature from near the main city center, or interpolated based on distance from that to the next data point. The National Weather Service has fewer of these than maybe Accuweather has access to. But it's not really important to the services they provide.
It seems that Apple maybe used to use Weather Channel data and
Re: (Score:2)
Well it's a good thing they bought DarkSky and ruined it then.
Re: (Score:2)
That is what weather models are there to do, take the physics knowledge and apply it to the existing weather data for how things are right now, and then put it into motion.
Re: (Score:2)
Our weather reports we used to get were from the Air Force (as they have a base on the island) and those were pretty accurate, but when the big Boston stations would give the Cape weather reports, they were more often wrong than right, routinely when it was pouring down rain the Boston stations would report Sunny and nice, and urge people to go enjoy the
Re: (Score:2)
Re: (Score:2)
Yeah, it makes sense that they might do this sort of thing, but... well. I had never caught them doing it before so naively thought that they were actually reporting the actual weather as best they cou
Re: (Score:2)
When you have a desired result, then you will always have people or AI that will come up with a "hallucination" to support that result. If you go based on what you actually have data on, then there will be no hallucination, and things will change as new information gets added. When it comes to the weather, the real key is to have sources, and the more of them, the better. Weather Underground has been good about that, because you have more sources of weather data. Forecasts will still be prone to bei
Re: About damned time (Score:2)
Re: (Score:2)
Weather prediction is actually REALLY good. But the tiny details matter a lot.
Weather models can accurate predict a huge storm rolling in. In general, the storm happens exactly as described and roughly at the same intensity. But for your specific street address, the storm may miss you to the north or south because of the very small details. Of course, a large scary cloud can either develop into a major storm with tornadoes or do nothing at all based on very tiny details that are variable right up to the
Re: (Score:2)
..., which is why so many consumers get irritated as hell when so many “experts” try to do so. And fail. Consistently.
I find it funny when a weather "expert" predicts a 100% chance of rain a day or two before the forecast rain. 100% means something very different to them, it just has to.
Re: (Score:2)
The more data sources they have, the more accurate the weather predictions will become. Adding weather stations in areas with a lot of hills and valleys would help come up with better predictions.
Re: (Score:2)
This seems like an application tailor made for AI: weather pattern matching based on historical data. As I drive a lot, even in the winter, I look forward to increased accuracy of weather reports.
Weather models have been using machine learning for various parts since forever, including things like detail parts and cloud coverage calculations. Sounds like what google is proposing as well.
The article writes the issue:
"While new machine-learning techniques that predict weather by learning from years of past data are extremely fast and efficient, they can struggle with long-term predictions"
The AI crowd these days is trying to apply neural networks and transformers and such for these models. They work o
Re: (Score:2)
The ML models being discussed here used to be called "ad-hoc" models, that's a technical term which means you put in random pieces of information you've got available and hope for the best. Another term for "ad-hoc" is "data-driven". All of these approaches share the same premise: if you mash things togeth
Re: (Score:2)
That was my understanding as well, all they're doing is replacing some bits of the model which were previously doing some deterministic calculations with some (hopefully somewhat) intelligent guesswork by AI, because these bits of the model weren't functioning well for whatever reason. Seems like a pragmatic decision to make up for the currently weak part of the model but hardly any kind of technological advance worth writing about. Ultimately what we want is a deterministic model that's capable of making a
Re: (Score:2)
I wonder how well it will cope with climate change. If it is trained on historic data then it might not be prepared for the changes we are already seeing, and their magnitude is only going to get larger.
Re: (Score:2)
On the other hand, it may pick out trends in the data that build in some understanding of climate change, as it's been happening steadily enough for us to see the rise in temperatures and increase in severity of weather events.
Re: (Score:2)
This seems like an application tailor made for AI: weather pattern matching based on historical data. As I drive a lot, even in the winter, I look forward to increased accuracy of weather reports.
Meteorologists that never look out a window really, REALLY suck at predicting the weather. Around here, you get the "head in a computer" meteorologists with a track record around 30-50% accurate only looking at their forecasts for precipitation and temperature. Then you have the "half the time I'm outside" guys, who usually manage to get closer to 70-80% accurate. How is AI, by definition an artificial head in a computer, going to do better than humans who never look at the actual systems they are studying?
Re: (Score:2)
In North County San Diego, weather prediction apps are useless because at the beach vs. 1 mile inland are completely different. You literally have to look out the window to see where the marine layer is breaking, and then use your own brain to guess where it will drizzle, partial clouds, be sunny, be windy etc.
Re: (Score:2)
Hi Aaron,
If I want to know what weather is going to be like, I look at this first: https://earth.nullschool.net/ [nullschool.net]
This will not show you if you are experiencing rainfall currently, but it will be obvious that rain or not is expected in your area. Between that URL and the local weather maps, I am 100% prepared for any weather conditions.
I found that URL from here at Slashdot. How cool is that?
Re: (Score:2)
Re: (Score:2)
I don't really know how to read it either. What I assume is being shown is the amount of green is representative of the amount of moisture in the air. The lines that are green seem to move in relation to the direction of wind speed.
No green, no moisture, but could still be winds.
I look at the "local" radar after looking at the map I shared with you and the local radar makes a lot more sense with this map as the overview of prevailing conditions. For example, there was light green lines over my area recently
2 years from now : Google's weather app cancelled (Score:3)
Just wait for it!
Re: (Score:2)
How big is the trained model? (Score:2)
I'm not an AI expert but my impression after looking at some LLM code is that the code tends to be quite small, it's the trained model that the code acts on that's large. So 5500 lines of code vs NOAA might not really be apples to apples here?
Re: How big is the trained model? (Score:2)
So few LoC! (Score:2)
It's apparently very important that this model is small: "a machine-learning model like Google’s GraphCast can run on less than 5,500 lines of code, compared with the nearly 377,000 lines required for the model from the National Oceanic and Atmospheric Administration,". Because we all know that it's the lines of code that are performance drags, not what those lines do, right?
Re: So few LoC! (Score:2)
Will the predictions have 6 fingers? (Score:1)
Eom
Forecast (Score:3)
That's great and all (Score:2)
But can you PLEASE offer a radar data overlay with projected storm path into Android Auto/Maps? When driving, some of us weather nerds would like to visualize emergent severe weather around us and how it may impact our safety or ETA.
Re: (Score:2)
You'd think a tornado warning should at least give you a comparison between your current trajectory and the trajectory of a tornado. But the position in the official alerts is usually descriptive (3 miles east of Townname) and not GPS coordinates and vector.
Re: (Score:2)
>You'd think a tornado warning should at least give you a comparison between your current trajectory and the trajectory of a tornado. But the position in the official alerts is usually descriptive (3 miles east of Townname) and not GPS coordinates and vector.
Maybe because it would be fucking useless? RADAR images take time, most update on intervals measured in minutes, and tornadoes are notorious for moving semi erratically to extremely erratically. Some / many even do loops. In minutes or less.
Plus you
"Weather+"? (Score:2)
What they're doing sounds really cool - but if it becomes an actual Google-branded product/service, don't get too attached to it...
Current forecasts are garbage (Score:2)
Current forecasting is garbage, and I believe the reason is that they are overpromising on the granularity of the data. When you're forecasting hourly and even minutely data, it *seems* like it should be accurate because they are stating it in such precision. In this case it's GIGO, and the precision doesn't actually exist.
DarkSky was very bad about this. Apple bought them up and now use that data in their built-in iOS app, and the data is still garbage. Based on how DarkSky claimed they were going to work,
Snow tomorrow! (Score:2)
Can't wait for the hallucinations to begin.
Exciting (Score:1)