I'm working on a research to evolve activation functions (using genetic algorithms) and I want to combine several activation functions together.. meaning if I have ReLU and Sin function, I could have Sin(ReLU(x)) or Sin(x)+Relu(x) as an activation function.. But I don't want to create a nn.Module class for each of these AFs, I want it to evolve during the evolution process... is there any way to do so?
thanks!
