2

I am learning about discretization of the state space when applying reinforcement learning to continuous state space. In this video the instructor, at 2:02, the instructor says that one benefit of this approach (radial basis functions over tile coding) is that "it drastically reduces the number of features". I am not able to deduce this in the case of a simple 2D continuous state space.

Suppose we are dealing with 2D continuous state space, so any state is a pair $(x,y)$ in the Cartesian space. If we use Tile Coding and select $n$ tiles, the resulting encoding will have $2n$ features, consisting of $n$ discrete valued pairs $(u_1, v_1) \dots (u_n, v_n)$ representing the approximate position of $(x,y)$ in the frames of the $n$ 2-D tiles. If instead we use $m$ 2-D circles and encode using the distance of $(x,y)$ from the center of each circle, we have $m$ (continuous) features.

Is there a reason to assume that $m < 2n$?

Furthermore, the $m$-dimensional feature vector will again need discretization, so it is unclear to me how this approach uses fewer features.

nbro
  • 39,006
  • 12
  • 98
  • 176
farhanhubble
  • 181
  • 4

0 Answers0