2

I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).

My question is (and I know this will/may be subjective) are the indicators for choosing to model some decision making problem as a single agent, treating all other factors/noise as part of the environment (MDP or some variant of it) vs multiple agents (a stochastic/Markov game)?

In the zero-sum/adversarial or pure cooperation setting where the goals of the agents conflict/assist each other, it is obvious that the multi-agent setting is the way to go. But suppose that there is no pure conflict/coordination.

nbro
  • 39,006
  • 12
  • 98
  • 176
David
  • 121
  • 3
  • Note that "Markov games" are already stochastic, so saying "stochastic Markov game" is a bit redundant. Markov games are sometimes called "stochastic games", so I changed "stochastic Markov game" to "stochastic/Markov game" in your post (because maybe that's what you meant originally anyway). – nbro Apr 07 '22 at 10:33
  • Thank @nbro, I understand that Markov games are stochastic but not all stochastic games are Markovian. I was referring to both cases where the Markov property may or may not apply. I should think there may be tricks for making general stochastic games (approximately) Markovian tho. – David Apr 07 '22 at 16:02

0 Answers0