1

In NEAT you can add a special bias input node that is always active. Regarding the implementation of such a node there is not much information in the original paper. Now I want to know how the bias node should behave, if there is a at all a consensus.

So the question is:

  • Do connections from the bias node come about during evolution and can be split for new nodes just like regular connections or does the bias node always have connections to all non-input nodes?
Tloy
  • 11
  • 3
  • From where do you have the information about the existence of a 'special bias input node that is always active'? As you said yourself, the original NEAT paper specified the algorithm without a bias in the nodes. Many modern implementations of NEAT however support bias, as does the most popular framework 'NEAT-Python', though NEAT-Python has bias in the regular nodes and this bias mutates just as the kernel weights in each node. – Paul Pauls May 04 '20 at 07:27
  • In the original paper they mention under 4.2 on page 112 that they used a network with bias to evaluate the XOR-case. – Tloy May 04 '20 at 08:06
  • You are right, I seem to have forgotten that NEAT has been defined with bias in mind. I somehow thought that adding bias consideration was an addition of modern algorithm implementations. Either way, I just read up on all 23 mentions of bias in the NEAT PhD thesis, all of them seem to be an addition to the kernel of a neuron. From where do you have the information about 'special bias nodes'? – Paul Pauls May 04 '20 at 08:21

1 Answers1

0

To answer my own question: According to the NEAT users page Kenneth O. Stanley talks about why the bias in NEAT is used as an extra input neuron:

Why does NEAT use a bias node instead of having a bias parameter in each node?

Mainly because not all nodes need a bias. Thus, it would unnecessarily enlarge the search space to be searching for a proper bias for every node in the system. Instead, we let evolution decide which nodes need biases by connecting the bias node to those nodes. This issue is not a major concern; it could work either way. You can easily code a bias into every node and try that as well.

My best guess is therefore that the BIAS input is treated like any other input in NEAT, with the difference that it is always active.

Community
  • 1
  • 1
Tloy
  • 11
  • 3