Markov Decision Process Clarification

Question

I'm in the process of implementing Value Iteration for a homework assignment. It's coming along nicely but I'm confused about a certain part, specifically the line indicated below

//(taken from http://aima.cs.berkeley.edu/python/mdp.html)
def value_iteration(mdp, epsilon=0.001):
  "Solving an MDP by value iteration. [Fig. 17.4]"
  U1 = dict([(s, 0) for s in mdp.states])
  R, T, gamma = mdp.R, mdp.T, mdp.gamma
  while True:
      U = U1.copy()
      delta = 0
      for s in mdp.states:
          U1[s] = R(s) + gamma * max([sum([p * U[s1] for (p, s1) in T(s, a)])
                                    for a in mdp.actions(s)])
          delta = max(delta, abs(U1[s] - U[s])) //*****THIS LINE**************//
      if delta < epsilon * (1 - gamma) / gamma:
          return U

I understand the point of the line in general but do I need to compare the updated utility to the old version or to the last state updated or what? Currently what I have seems to be working (mostly :P) but I was confused because other versions of the algorithm such as this one have that k <- k + 1 and ∀s |Vk[s]-Vk-1[s]| < θ which makes me think I'm doing it wrong.

Here is my code:

grid = [[0,0,0],[0,None,0],[0,0,0],[0,-1,1]]

gamma = .9
epsilon = 0.001 #difference threshold
count = 0
while(True):
   delta = 0
   #walk through the set of states

   i = 0   
   while(i < 4):
       j= 0
       while(j < 3):
          if(grid[i][j] == None or grid[i][j] == 1 or grid[i][j] == -1):
              j = j +1
              continue
          temp =  grid[i][j] 
          grid[i][j] = -0.04 + (gamma * bellman(grid, i, j))
          delta = max(delta, abs(grid[i][j] - temp))

          j = j+1
      i = i+1

  if (delta < (epsilon * (1 - gamma) / gamma)): 
      break

and I get an output of:

0.5094143831769762  0.6495863449484525  0.795362242280654    1
0.39850269350488843 None                0.48644045209498593 -1
0.29643305379491625 0.25395638075084487 0.344787810489289    0.12994184490884678