Reducing the Computational Cost of Deep Reinforcement Learning Research (googleblog.com) 5
Pablo Samuel Castro, Staff Software Engineer at Google Research, writes on Google AI blog: It is widely accepted that the enormous growth of deep reinforcement learning research, which combines traditional reinforcement learning with deep neural networks, began with the publication of the seminal DQN algorithm. This paper demonstrated the potential of this combination, showing that it could produce agents that could play a number of Atari 2600 games very effectively. Since then, there have been several approaches that have built on and improved the original DQN. The popular Rainbow algorithm combined a number of these recent advances to achieve state-of-the-art performance on the ALE benchmark. This advance, however, came at a very high computational cost, which has the unfortunate side effect of widening the gap between those with ample access to computational resources and those without.
In "Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research," to be presented at ICML 2021, we revisit this algorithm on a set of small- and medium-sized tasks. We first discuss the computational cost associated with the Rainbow algorithm. We explore how the same conclusions regarding the benefits of combining the various algorithmic components can be reached with smaller-scale experiments, and further generalize that idea to how research done on a smaller computational budget can provide valuable scientific insights.
In "Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research," to be presented at ICML 2021, we revisit this algorithm on a set of small- and medium-sized tasks. We first discuss the computational cost associated with the Rainbow algorithm. We explore how the same conclusions regarding the benefits of combining the various algorithmic components can be reached with smaller-scale experiments, and further generalize that idea to how research done on a smaller computational budget can provide valuable scientific insights.
Recent advances in AI NOT mainly computational (Score:3)
Some have a habit whenever any new advance is announced in areas like neural networks to scoff that all the underlying theory has been known for decades, and recent advances are just because of the ability to crunch more data (or, related, greater availability of data). This is nonsense. While availability of massive data sets, and the ability to process them is very helpful, the primary reason for the increasing capabilities is cutting edge research, such as the examples cited in the FA. Nevertheless, bringing down the cost of using AI effectively is critical in making its use ubiquitous, as I believe is just around the corner.
Re: (Score:2)
The advances in AI in the last decade have been due to figuring out how to train deep models, a purely algorithmic advancement. This story is about a kind of a wrapping up of tips and tricks for training deep reinforcement models, but they're also algorithmic.
I think the availability of compute and large datasets has hampered progress. The data processing inequality makes it clear that no matter how smart your system is, you can't create information that isn't already in the data. The last five years have
Re:Recent advances in AI IS mainly computational (Score:2)
Where is the cutting edge research? It's all incremental research. Nothing fundamental has come out in a very long time. I think closest to fundamental novelty has been Long Term Short Term perceptrons, perhaps.. a while back.. But even those are really parametric.. they are mostly static.
The set theory notations used and the libraries, such as TensorFlow, frame neural nets in highly constrained manner that makes ground-breaking research very difficult, if not outright impossible.
There is nothing but in