Q value for the absorbing state

Question

\begin{equation}
Q_{t+1}(s_t,a_t) = Q_{t}(s_t,a_t) +\alpha
(R_{t+1} + \gamma * \max(Q_t(s_{t+1}, a)) - Q_t(s_t, a_t))
\end{equation}

In above equation,there is a term max(Q_t(s_{t+1},a)) Now say after you take an action in state s_t resulting in s_{t+1}. There are no available moves in s_{t+1}. The game has ended in draw, What is this max(Q_t(s_{t+1},a)) then?

Pablo EM · Answer 1 · 2016-06-13T10:34:32.233

2

The value of terminal (aka absorbing) states are 0 by definition in V and Q functions, as it can be read in Section 3.7 of Rich Sutton's book:

edited Jun 13 '16 at 10:34

answered Jun 13 '16 at 09:17

Pablo EM

6,190
3
29
37

Can you please mention the definition. – Abhishek Bhatia Jun 13 '16 at 09:19
Thanks, if possible please include the explicit definition in the answer. – Abhishek Bhatia Jun 13 '16 at 09:33
@AbhishekBhatia if your question was correctly answered, could you please mark the answer as accepted? Thanks. – Pablo EM Jun 16 '16 at 06:49

Q value for the absorbing state

1 Answers1