4

Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?

For example:

  • I use level 1 experiences, my model $p$ is fitted to learn how to play that level.

  • I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.

nbro
  • 39,006
  • 12
  • 98
  • 176
Chukwudi
  • 349
  • 2
  • 7
  • 1
    Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: https://ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question. – nbro Jul 27 '20 at 13:17
  • This question is very related to https://ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN. – nbro Jul 28 '20 at 13:10

1 Answers1

5

You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this article.

pedrum
  • 313
  • 1
  • 13
  • 1
    This was incredibly helpful, Thank you – Chukwudi Jul 27 '20 at 12:20
  • 1
    I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution? – Chukwudi Jul 27 '20 at 12:23
  • @Chukwudi I'm not really sure, sorry. – pedrum Jul 27 '20 at 13:45