Researchers Build AI That Builds AI (quantamagazine.org) 30
By using hypernetworks, researchers can now preemptively fine-tune artificial neural networks, saving some of the time and expense of training. From a report: Artificial intelligence is largely a numbers game. When deep neural networks, a form of AI that learns to discern patterns in data, began surpassing traditional algorithms 10 years ago, it was because we finally had enough data and processing power to make full use of them. Today's neural networks are even hungrier for data and power. Training them requires carefully tuning the values of millions or even billions of parameters that characterize these networks, representing the strengths of the connections between artificial neurons. The goal is to find nearly ideal values for them, a process known as optimization, but training the networks to reach this point isn't easy. "Training could take days, weeks or even months," said Petar Velickovic, a staff research scientist at DeepMind in London. That may soon change.
Boris Knyazev of the University of Guelph in Ontario and his colleagues have designed and trained a "hypernetwork" -- a kind of overlord of other neural networks -- that could speed up the training process. Given a new, untrained deep neural network designed for some task, the hypernetwork predicts the parameters for the new network in fractions of a second, and in theory could make training unnecessary. Because the hypernetwork learns the extremely complex patterns in the designs of deep neural networks, the work may also have deeper theoretical implications. For now, the hypernetwork performs surprisingly well in certain settings, but there's still room for it to grow -- which is only natural given the magnitude of the problem. If they can solve it, "this will be pretty impactful across the board for machine learning," said Velickovic.
Boris Knyazev of the University of Guelph in Ontario and his colleagues have designed and trained a "hypernetwork" -- a kind of overlord of other neural networks -- that could speed up the training process. Given a new, untrained deep neural network designed for some task, the hypernetwork predicts the parameters for the new network in fractions of a second, and in theory could make training unnecessary. Because the hypernetwork learns the extremely complex patterns in the designs of deep neural networks, the work may also have deeper theoretical implications. For now, the hypernetwork performs surprisingly well in certain settings, but there's still room for it to grow -- which is only natural given the magnitude of the problem. If they can solve it, "this will be pretty impactful across the board for machine learning," said Velickovic.
Tuning? (Score:4, Interesting)
Re:Tuning? (Score:4, Interesting)
Most machine learning articles use so much jargon, it's difficult to tell what's a breakthrough and what's bullshit.
Re: (Score:2)
Most machine learning articles use so much jargon, it's difficult to tell what's a breakthrough and what's bullshit.
It's safe to assume that any given paper is more bullshit than breakthrough.
On the use of jargon, this one reads a lot like a student paper. Jargon should be used only when it adds clarity or is otherwise necessary. Language overall is a bit unusual, with a lot of needless adjectives. They also hype their results in the paper, which is a red flag.
I really don't like their use of 'experiment' and 'hypothesis', but this isn't unusual for CS papers.
Just a first impression. The paper is here [arxiv.org], if anyone want
Watching It All Unfold... (Score:3)
Hmmmm.
"Hungry for data and power", "carefully tuning values" and "overlord".
Yeah, see, those are three phrases and/or concepts that have absolutely no place in the context of artificial intelligence.
It's almost as if they're taking their cues FROM Hollywood.
Machines making machines... (Score:3)
what could go wrong with that?
Just remember the Real Rule of Robotics, "Humans are fascinating and we want to make sure they are having a good time"
That's nothing (Score:2)
I built an AI that builds Ais that build AIs. And it does it in 7 minutes while you do your Abs.
"performs surprisingly well in certain settings" (Score:2)
Re: (Score:2)
Pretty much yes. A lot of research has degraded from actually being useful to giving the appearance of being useful.
But Do They... (Score:3)
Researchers Build AI That Builds AI
But do they have an AI to build that?
Re: (Score:2)
They don't even have the first one. Pop sci AI article headline is bullshit. Shocking, I know.
Not just "enough data and processing power" (Score:2)
Modern machine learning benefitted from two recent advances:
* GPU-based neural network processing
* Faster learning algorithms, so networks could converge in a reasonable time
See here [machinelea...owledge.ai] for a historical timeline.
GPU use started around 2008.
Theoretical work that sped up learning in 2006 and 2011.
Re: (Score:2)
Prior to that there wasn't really any good algorithm for this problem, other than evolving the network with gradient descent type techniques (sure that is technically an algorithm, but more of a brute force thing of trying random variations vs an actual update formula)
Yo dawg (Score:2)
I heard you liked AI so I added AI to your AI.
At least we will soon be able to put the crackpot theories of paranoid futurists to the test. Of course I hope those guys were all crackpots ...
Archer: "Do you want Skynet?" (Score:3)
Re: (Score:1)
Deja Vu (Score:2)
Re: (Score:3)
Westworld is a closer approximation. Robots designed better robots. Several generations later, humans no longer understood how they worked.
Think how hard it is to detail how biological life works. Now imagine how intricate and specialized and interleaved sections of a giant AI brain might get.
Re: (Score:2)
How do you get a really f'd up AI? (Score:1)
Use an AI to build an AI to build an AI to build an AI... it's turtles all the way down.
What used to take humans days, weeks, months, to royally screw up, the AI can do so in milliseconds.
Faster than you can pull the plug before the genie escapes the bottle.
The clowns of AI are like the early scientists of nuclear energy. Only difference, the fallout from the nuclear energy scientists was somewhat contained.
So finally the Bitch-Stewie of AI was invented. (Score:2)
And now he's going to build the Bitch-Brian of AI.
Yo dawg, I heard you like trained AIs (Score:2)
... so I trained an AI to train AIs.
Do you want Skynet? (Score:2)
Because that's how you get Skynet ...
Seems totally legit (Score:1)
So... (Score:2)
"It's an anti-terminator... terminator?"