3

This is an ambitious question from a Wolfram Science Conference: Is there such a thing as a network analog of a recursive function? Maybe a kind of iterative "map-reduce" pattern? If we add interaction to iteration, things become complicated: continuous iteration of a large number of interacting entities can produce very complex results. It would be nice to have a way of seeing the consequences of the myriad interactions that define a complex system. Can we find a counterpart of a recursive function in an iterative network of connected nodes which contain nested propagation loops?

One of the basic patterns of distributed computation is Map-Reduce: it can be found in Cellular Automata (CA) and Neural Networks (NN). Neurons in NN collect informations through their synapses (reduce) and send it to other neurons (map). Cells in CA act similar, they gather informations from their neighbors (reduce), apply a transition rule (reduce), and offer the result to their neighbors again. Thus >if< there is a network analog of a recursive function, then Map-Reduce is certainly an important part of it. What kind of iterative "map-reduce" patterns exist? Do certain kinds of "map-reduce" patterns result in certain kinds of streams or even vortices or whirls? Can we formulate a calculus for map-reduce patterns?

0x4a6f4672
  • 27,297
  • 17
  • 103
  • 140

1 Answers1

2

I'll take a stab at the question about recursion in neural networks, but I really don't see how map-reduce plays into this at all. I get that neural network can perform distributed computation and then reduce it to a more local representation, but the term map-reduce is a very specific brand of this distributed/local piping, mainly associated with google and Hadoop.

Anyways, the simple answer to your question is that there isn't a general method for recursion in neural networks; in fact, the very related simpler problem of implementing general purpose role-value bindings in neural networks is currently still an open question.

The general principle of why things like role-binding and recursion in neural networks (ANNs) are so hard is that ANNs are very interdependent by nature; indeed that is where most of their computational power is derived from. Whereas function calls and variable bindings are both very delineated operations; what they include is an all-or-nothing affair, and that discreteness is a valuable property in many cases. So implementing one inside the other without sacrificing any computational power is very tricky indeed.

Here is a small sampling of papers that try there hand at partial solutions. Lucky for you, a great many people find this problem very interesting!

Visual Segmentation and the Dynamic Binding Problem: Improving the Robustness of an Artificial Neural Network Plankton Classifier (1993)

A Solution to the Binding Problem for Compositional Connectionism

A (Somewhat) New Solution to the Binding Problem

zergylord
  • 4,368
  • 5
  • 38
  • 60
  • Interesting answer, although I am not sure what the binding problem has to do with the question. – 0x4a6f4672 Jun 30 '11 at 08:11
  • I know that map-reduce is a very specific term mainly associated with Google and Hadoop. I think they maybe have discovered a very fundamental thing in distributed systems, a [basic abstraction of interactive computation](http://bit.ly/myDCUJ). In neural networks, we have map-reduce patterns on many levels: from stretching and folding of information between neurons at the bottom, merging and splitting of neural assemblies in the middle, to gathering information and reducing it to a decision at the top. There is something very fundamental to it. – 0x4a6f4672 Jun 30 '11 at 21:50