Non-recursive traversal of nested list - building a similar nested list in Python

Question

I need to walk a nested list, process every non-list item with str() and return similar list keeping the structure. With recursion it would be rather easy, but I need to do this iterative way. Below is my attempt with a while loop:

def myiter(e):
    a = [e] # initial list
    c = [[]] # final result
    get_last = lambda x: x[len(x)-1] # get ref to the final sublist
    l = get_last(c)
    while a:
        b = a.pop(0)
        if isinstance(b, list):
            # if there are more items to process in the original list
            if a:
                a = b + a
            # else extend original list to process sublists
            else:
                a.extend(b)
            # make a new sublist ref
            l = get_last(c)
            c.append([])
        else:
             # walk and process every item in the nested list
             l.append(str(b))
    return c

There are several problems with this as the output will show:

myiter([1, [2, [3, 4], 5]]) # [['1'], ['2'], ['3', '4', '5'], []]

Desired result is:

['1', ['2', ['3', '4'], '5']]

Is there a simple iterative way to do the task in Python?

Answer 1

This seems to work:

def stringify(a):
    a = a[:]                           # Make copy of what was passed in.
    res = []                           # Initialize result list.
    my_stack = []                      # Initialize our own LIFO stack.
    while (a or my_stack):                            # While a or my_stack is non-empty
        if (a):
            elem = a.pop(0)
            if (not isinstance(elem, list)):          # If popped elem is not a list
                res.append(str(elem))                 # Append stringified elem to res
            else:
                my_stack.append((a, res))           # Push some stuff, to resume working upon later.
                a = elem                            # Let's start iterating on this inner list
                res = []                            # This inner list needs a clean res, to start with.
        else:                                       # my_stack is non-empty
            a, res_prev = my_stack.pop()   # Pop some stuff, to resume, work on outer list
            res_prev.append(res)           # First, append our just-completed inner list.
            res = res_prev
    return res

Output:

a = [1, [2, [3, 4], 5]]
stringify(a)
['1', ['2', ['3', '4'], '5']]

Passed the Following Test Cases:

a = [1, [[[2]]]]
a = [[[1]], 2]
a = [1, [[2]]]
a = [1, [2, [3, 4], 5], [6, [7, [8]]], 9]
a = [1, [2, [3, 4], 5]]
a = [1, 2, 3, 4, 5]

Some Notes on How This Works:

1. If the pop on our list a produces an integer, we just append the stringified integer to res
2. If the pop on our list a produces an inner list, we need to start processing that inner list, before we process the elements occurring after that inner list. After processing the inner list, we will have to come back to the remaining unpopped elements of a).
3. Whenever we detect that our current list a has become empty, our res would be pointing to the equivalent stringified list, and it is time for us to append our res to whatever may be its (stringified) outer list
4. To process each encountered inner list, we adopt that inner list as our new a (the assignment a = elem), and an empty list as our new res (res = []). Before we do that, we need to push onto the stack our current a and our current res
5. Why LIFO? Well, look at it this way: Whatever was pushed last onto my_stack represents the the immediate outer list of whatever list we are currently processing (a).

Answer 2

Why not recursion? Processing recursive data structures using recursive procedures is natural and straightforward. Converting a recursive process to an iterative one doesn't have to involve cloning the input, creating a stack, or other intermediate values. Your brain can be free from such crippling complexities -

def first (a = []):
  return a[0]

def rest (a = []):
  return a[1:]

def myiter (a = []):
  # base: empty a
  if not a:
    return []
  # inductive: non-empty a, first elem is list
  elif isinstance(first(a), list):
    return [ myiter(first(a)) ] + myiter(rest(a))
  # inductive: non-empty a, first elem is non-list
  else:
    return [ str(first(a)) ] + myiter(rest(a))     

print(myiter([1, [2, [3, 4], 5]]))

Of course it makes sense to make str a parameter of the function, f -

def myiter (f, a = []):
  # base: empty a
  if not a:
    return []
  # inductive: non-empty a, first elem is list
  elif isinstance(first(a), list):
    return [ myiter(f, first(a)) ] + myiter(f, rest(a))
  # inductive: non-empty a, first elem is non-list
  else:
  return [ f(first(a)) ] + myiter(f, rest(a))

Now you can deep map str like you want to -

print(myiter(str, [1, [2, [3, 4], 5]]))
# ['1', ['2', ['3', '4'], '5']]

Or use any function of your choosing -

def square (x):
  return x * x

print(myiter(square, [1, [2, [3, 4], 5]]))
# [1, [4, [9, 16], 25]]

Are you trying to avoid recursion because of a stack limit? If you make it tail-recursive -

def identity (x):
  return x

def myiter (f, init = []):
  def run (a = init, then = identity):
    if not a:
      return \
        then([])
    # inductive: non-empty a, first elem is list
    elif isinstance(first(a), list):
      return \
        recur(first(a), lambda l: \
          recur(rest(a), lambda r: \
            then([ l ] + r)))
    # inductive: non-empty a, first elem is non-list
    else:
      return \
        recur(rest(a), lambda r: \
          then([ f(first(a)) ] + r))
  # loop inner function
  return loop (run)

Then implement a generic loop which converts the recursive call stack to an iterative sequence -

def recur (*values):
  return (recur, values)

def loop (f):
  acc = f ()
  while type(acc) is tuple and acc[0] is recur:
    acc = f(*acc[1])
  return acc

The output is the same, but now myiter can accept an array of any nesting limit. Recursion without restrictions; what a beautiful thing -

print(myiter(str, [1, [2, [3, 4], 5]]))
# ['1', ['2', ['3', '4'], '5']]

print(myiter(square, [1, [2, [3, 4], 5]]))
# [1, [4, [9, 16], 25]]

View this program and verify the results in your own browser using repl.it.

---

Why can't you use recursion? - Jab

@Jab, Three reasons: First maximum recursion limit is often hit on one of my app, second performance concerns althought map might be head to head with iterative method, thirdly just for training and studying these two different styles of coding. – MarkokraM

So you haven't hit a recursion limit but you're worried that your program will? It's better to understand actual limitations instead of writing programs around ghosts. In a simplified implementation using generators, notice that recursion only happens when a level of nesting is encountered. Even if you left this implementation as-is, this can support lists of any length and nesting levels up to the stack limit, where the default is probably around 1,000. This means the only data input that would blow your stack is one that is nested 1,000 times or more. It's probably safe to leave this program be until an actual limitation is reached.

def square (x):
  return x * x

def myiter (f, init = []):
  def gen (a):
    for x in a:
      if isinstance(x, list):
        yield list(gen(x)) # recursion
      else:
        yield f(x)
  return list(gen(init))

print(myiter(str, [1, [2, [3, 4], 5]]))
# ['1', ['2', ['3', '4'], '5']]

print(myiter(square, [1, [2, [3, 4], 5]]))
# [1, [4, [9, 16], 25]]