I tried DoubleDQN and DQN algorithm on gym NChain game and realized that the performance of DoubleDQN was not more stable or better than DQN.
I set batch size of the training after each action taken to be 1. May I know this is the reason of DoubleDQN not outperforming DQN?
It looks like both of your networks learned nothing or the environment doesn't cause optimistic in Q value. The original result of the n chain is 3643. A Bayesian Framework for Reinforcement Learning
In atari, Dobule DQN outperforms DQN in most of the games but have worse results in a few ones.
