I am trying to implement an MDP for the first time. Each state is a tuple of four variables. I want to implement a transition table, that maps each state and action to the next state.
transition_model = [tuple(x) for x in np.zeros((self.num_of_states*self.num_of_actions, self.num_of_states,9),int)]
When I try to :
transition_model[ct][x][y][p][x][ct+1][x+1][y+1][p] = 1
I get an error saying :
IndexError: invalid index to scalar variable.
Can anyone tell me how to fix this? Is there any better way to implement such an MDP. Thank you so much!
