4

I just read that OpenAI's ES uses Adam: "OpenAI’s ES is denoted as “OptimES” (since it uses Adam optimizer)"?? I verified they are correct using the link they posted, (see es_distributed/Optimizers.py). But I don't understand how because the paper says they are using Evolution Strategies as a Scalable Alternative to Reinforcement Learning, which is black box optimization (like most ES's)... So how on earth is Adam used, given this is black box?

nbro
  • 39,006
  • 12
  • 98
  • 176
profPlum
  • 360
  • 1
  • 9

1 Answers1

0

OpenAI's ES updates parameters using learning rate. It may remain constant, decaying or changed in different ways. You can for example use momentum that's commonly used in stochastic gradient descent. So Adam is used to adjust learning rates per parameters.

derjack
  • 106
  • 1
  • 7
  • Oh that's interesting. Just checking though you're sure about this? Could you add some citations? – profPlum May 12 '22 at 17:38
  • 1
    Not citations per se, but look at the code in [github](https://github.com/hardmaru/estool/blob/master/es.py#L331) from [Visual Guide to Evolution Strategies](https://blog.otoro.net/2017/10/29/visual-evolution-strategies/). – derjack May 12 '22 at 18:53