Questions tagged [bootstrapping]

In Barton and Sutton's book, Reinforcement Learning: An Introduction (2nd edition), an expression, on page 289 (equation 12.2), introduced the form of the $\lambda$-return defined as follows $$G_t^{\lambda} = (1-\lambda)\sum_{n=1}^{\infty}…

In the text book of Sutton and Barto on page 152 they introduce the n-step Tree Backup algorithm, where the tree-backup n-step return is defined via $$ G_{t:t+n} = R_{t+1} + \gamma \sum_{a \neq A_{t+1}} \pi(a | S_{t+1})Q_{t+n-1}(S_{t+1}, a) + \gamma…

The following quote is taken from the beginning of the chapter on "Approximate Solution Methods" (p. 198) in "Reinforcement Learning" by Sutton & Barto (2018): reinforcement learning generally requires function approximation methods able to handle…

I have a question about how to update the Q-function in Q-learning and SARSA. Here (What are the differences between SARSA and Q-learning?) the following updating formulas are given: Q-Learning $$Q(s,a) = Q(s,a) + \alpha (R_{t+1} + \gamma…