What are some (good) online courses for deep reinforcement learning?
I would like the course to be both programming and theoretical. I really liked David Silver's course, but the course dates from 2015. It doesn't really teach deep Q-learning at this time.
Let me first say that deep RL is just the combination of RL with deep learning. So, if you study RL and deep learning, then studying deep RL should be straightforward. For this reason, this answer will point the reader to potentially useful courses on RL (also because there aren't many free courses completely dedicated to deep RL), which have at least one lesson on deep RL or function approximation. I have only followed the course by Isbell and Littman and partially the course by David Silver, so I can't ensure you that the other courses are good, but I found these two useful, although not perfect.
| Title | Instructor(s) | Focus on deep RL? | Topics | Free |
|---|---|---|---|---|
| Reinforcement Learning | Charles Isbell, Michael Littman | No | TD learning, convergence, function approximation, POMDP, options, game theory | Yes |
| Introduction to Reinforcement Learning with David Silver | David Silver | No | MDPs, planning, dynamic programming, model-free prediction and control, function approximation, policy gradients, exploration and exploitation | Yes |
| CS234: Reinforcement Learning Winter 2020 | Emma Brunskill | No | See the course schedule; lesson 6 is about DRL | Yes |
| Reinforcement Learning | NPTEL | No | Bandits, MDPs, policy gradients, dynamic programming, TD learning, function approximation, hierarchical RL, POMDP | Yes |
| Reinforcement Learning in the Open AI Gym | Phil Tabor | ? | SARSA, double Q-learning, Monte Carlo methods, Q-learning | Yes |
| Advanced Deep Learning & Reinforcement Learning | DeepMind | No | video 14 discusses DRL topics | Yes |
| Advanced AI: Deep Reinforcement Learning in Python | Udemy | Yes, it seems | ? | No |
| Machine Learning: Beginner Reinforcement Learning in Python | Udemy | ? | ? | No |
| Deep Reinforcement Learning 2.0 | Udemy | ? | ? | No |
| Modern Reinforcement Learning: Deep Q Learning in PyTorch | Phil Tabor (Udemy) | ? | ? | No |
| Modern Reinforcement Learning: Actor-Critic Methods | Phil Tabor (Udemy) | ? | ? | No |
In any case, if you are familiar with RL and deep learning topics, I encourage you to directly read the DQN papers (both by DeepMind folks)
Of course, deep RL isn't just DQN, but these are two very important papers that you should read. Other key papers on deep RL can be found here.
Note that, depending on your experience with and knowledge of RL and DL, you may require a few iterations to fully understand these papers, but this applies every time you need to read a research paper.
For the programming part I suggest this YouTube channel by Phil Tabor (he also has a website: neuralnet.ai. I found his videos really useful while I was attending reinforcement learning classes at the uni. He covers basic algorithms like value iteration and policy iteration and also more advanced like deep q learning, covering all main python libraries (Keras, tensorflow, pytorch). Hope it will help you as well!
