4

In the book "Reinforcement Learning: An Introduction" (2018) Sutton and Barto explain, on page 221, a form of tile coding using hashing, to reduce memory consumption.

I have two questions about that:

  1. How can this approach reduce memory consumption? Doesn't it just depend on the number of tiles (you have to store one weight for each tile)?

  2. They state that there is only a "little loss of performance". In my understanding, the sense of tile coding (and coarse coding) is, that near-by states have many tiles in common and far-away states have only few tilings in common. With tilings "randomly spread throughout the state space" this isn't the case. How does this not influence performance?

nbro
  • 39,006
  • 12
  • 98
  • 176
F.M.F.
  • 311
  • 3
  • 7
  • you can look at the implementation [here](http://incompleteideas.net/tiles/tiles3.html) – Brale May 22 '19 at 08:47
  • I already saw the implementation and understand the concept, but not why it works! – F.M.F. May 22 '19 at 13:20
  • @ F.M.F. did you find an answer to this posted question? – user3489173 Apr 10 '22 at 21:32
  • 1
    @user3489173 In my understanding, the memory consumption is reduced, by assigning weights to tiles lazily and therefore only storing weights for parts of the state-action space that actually are reached. – F.M.F. Apr 11 '22 at 08:58

0 Answers0