Machine learning with a "function" that may have two or more outputs

Question

Let's say I want to model the sqrt function with a neural network. But for every input x, there are two answers sqrt(x) = y and sqrt(x) = -y. (But in reality, I don't know that I have the sqrt function - I just have a lot of data - so I don't know a priori if there are 0, 1, 2, or more answers y for every input x.) How can I get the correct distribution of y?

Answer 1

The question you should ask yourself, is how would you train such a neural network :

If you were thinking of giving x -> {sqrt(x),-sqrt(x)} as training example i, then that means that you know the number of outputs by looking at a single example so you should encode the fact that there are 2 outputs.

I you were thinking of giving examples that are sometimes x -> sqrt(x) and other time x -> -sqrt(x), then your neural network would be very hard to train as each gradient could change the weights in a completely different way. It would probably converge after many training examples, and would simply output the y it saw the most.

A better way of doing this would probably be to use something like a reward function in reinforcement learning. The function in that case would be f(x)=x^2 and would simply say if the output is correct. That would let your network train for both positive and negative outputs. So try to find a reward function for your problem!