Q-Learning policy doesn't agree with Value/Policy Iteration

Question

I am playing with pymdptoolbox. It has a built-in problem of forest management. It can generate a transition matrix P and R by specifying a state value for forest function (default value is 3). The implementation of Q-Learning, PolicyIteration and ValueIteration to find the optimal policy is straightforward. However by creating a slightly more complicated problem by changing the state to a bit larger value than 4 (from 5 onwards), only PI and VI return the same policy while QL cannot find the optimal policy. This is very surprising and puzzling. Can anyone help me understand why is this for QL in this package?

By looking at the raw code of QL (using epsilon-greedy), it seems it ties the probability with iteration number, i.e. prob = 1 - (1/log(n+2)) and the learning rate is (1/math.sqrt(n+2)). Is there any specific reason why tying probability/learning rate to the iteration number, instead of making them independent variables (the code itself can be modified easily though).

I think my biggest puzzle is to understand why QL fails to find the policy for a vanilla problem. Thanks.

from mdptoolbox.mdp import ValueIteration, QLearning, PolicyIteration
from mdptoolbox.example import forest

Gamma = 0.99

states = [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 30, 50, 70, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000]

compare_VI_QI_policy = [] # True or False
compare_VI_PI_policy = []

for state in states:

    P, R = forest(state)

    VI = ValueIteration(P, R, Gamma)
    PI = PolicyIteration(P, R, Gamma)
    QL = QLearning(P, R, Gamma)

    ## run VI
    VI.run()

    # run PI
    PI.run()

    # run QL
    QL.run()

    compare_VI_QI_policy.append(QL.policy == VI.policy)
    compare_VI_PI_policy.append(VI.policy == PI.policy)

print compare_VI_QI_policy
print compare_VI_PI_policy