1

I encountered such problem after I specified a differential evolution algorithm and an initial population of multiplied layer perceptron network. It requires to evolve a population of MLPs by DE. I tried to use Evolutionary package, but failed at this problem. I am just a beginner of julia. Can anyone help me with this problem? Or if there is any other way to implement a DE to evolve MLPs? Because I don't know much how to reuse codes if I don't see any similar example, I can't find any example of julia to evolve MLP by DE. The codes are attached as follow.

enter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description here //Here are the snippets of codes

begin
    features = Iris.features();
    slabels = Iris.labels();
    classes = unique(slabels)  # unique classes in the dataset
    nclasses = length(classes) # number of classes
    d, n = size(features)      # dimension and size if the dataset
end

//define MLP

model = Chain(Dense(d, 15, relu), Dense(15, nclasses))

//rewrite initial_population to generate a group of MLPs

begin
    import Evolutionary.initial_population
    function initial_population(method::M, individual::Chain;
                            rng::Random.AbstractRNG=Random.default_rng(),
                            kwargs...) where {M<:Evolutionary.AbstractOptimizer}
     θ, re = Flux.destructure(individual);
        [re(randn(rng, length(θ))) for i in 1:Evolutionary.population_size(method)]
    end
end

//define DE algorithm and I just used random parameters

algo2 = DE(
        populationSize=150,
        F=0.9,
        n=1,
        K=0.5*(1.9),
        selection = rouletteinv
        
    )
popu = initial_population(algo2, model)

//in the source code of Evolutionary.jl, it seems that to use optimize() function, I need to pass a constranit? I am not sure. I have tried every method of optimize function, but it still reported error. What's worse, I am not sure how to use box constraint, so I tried to use Nonconstranit constraint, but it still failed. I don't know how to set upper and lower bounds of box constraint in this case, so I don't know how to use it. and I tried to set a random box constraint to try to run optimize() function, but it still failed. error reported is in pitcure attached.

cnst = BoxConstraints([0.5, 0.5], [2.0, 2.0])
res2 = Evolutionary.optimize(fitness,cnst,algo2,popu,opts)

//so far what I do is simply define a DE algorithm, an initial population, a MLP network and there is a uniform_mlp(), which is used to deconstruct a mlp into a vector, perform crossover operator and reconstruct from them a new mlp

function uniform_mlp(m1::T, m2::T; rng::Random.AbstractRNG=Random.default_rng()) where {T <: Chain}
    θ1, re1 = Flux.destructure(m1);
    θ2, re2 = Flux.destructure(m2);
    c1, c2 = UX(θ1,θ2; rng=rng)
    return re1(c1), re2(c2)
end

//there is also a mutation function

function gaussian_mlp(σ::Real = 1.0)
    vop = gaussian(σ)
    function mutation(recombinant::T; rng::Random.AbstractRNG=Random.default_rng()) where{T <: Chain}  
            θ, re = Flux.destructure(recombinant)
        return re(convert(Vector{Float32}, vop(θ; rng=rng)))
    end
    return mutation
end
Meimei_Huang
  • 11
  • 2

1 Answers1

1

The easiest way to use this is through Optimization.jl. There is an Evolutionary.jl wrapper that makes it use the standardized Optimization.jl interface. This looks like:

using Optimization, OptimizationEvolutionary
rosenbrock(x, p) =  (p[1] - x[1])^2 + p[2] * (x[2] - x[1]^2)^2
x0 = zeros(2)
p  = [1.0, 100.0]
f = OptimizationFunction(rosenbrock)
prob = Optimization.OptimizationProblem(f, x0, p, lb = [-1.0,-1.0], ub = [1.0,1.0])
sol = solve(prob, Evolutionary.DE())

Though given previous measurements of global optimizer performance, we would recommend BlackBoxOptim's methods as well, this can be changed through simply by changing the optimizer dispatch:

using Optimization, OptimizationBBO
sol = solve(prob, BBO_adaptive_de_rand_1_bin_radiuslimited(), maxiters=100000, maxtime=1000.0)

This is also a DE method, but one with some adaptive radius etc. etc. that performs much better (on average).

Chris Rackauckas
  • 18,645
  • 3
  • 50
  • 81
  • thank you for helping me first, but I am still confused. in the offical documentation of Evolutionary,jl, I found a piece of code example to apply genetic evoltionary algorithm to MLP. It used Evolutionary.optimize(fitness, model, GA, opts) this method to get result. I thought I could use the same method, or others of optimize() function, to replace evolutionary algorithm, but failed. Though it is new to me to read source code and simply understand how to use by source code, since there is no other resouces to tell me how to use this package. – Meimei_Huang Dec 14 '22 at 14:39
  • As to Optimization package you mentioned, it seems more challenging for me, because it seems I also need to set box constraint to solve a problem? I read basic usage in this page, and I still don't know how to set constraint for a vector from flatten model https://docs.sciml.ai/Optimization/stable/tutorials/intro/ – Meimei_Huang Dec 14 '22 at 14:44
  • What do you mean by "constraint for a vector from flatten model"? What kind of constraints? IIRC, Evolutionary.jl only accepts box constraints, and you have to use other methods to use [the more advanced constraints](https://docs.sciml.ai/Optimization/stable/tutorials/constraints/). – Chris Rackauckas Dec 14 '22 at 15:52
  • In this example given by official documentation, here https://github.com/wildart/Evolutionary.jl/blob/cf3f2fbf5e1be13d6f5db23eabe5dbfab4d5b688/examples/MLP.ipynb it seems that 'res = Evolutionary.optimize(fitness, model, algo, opts)' this function allows to process a model of MLP by GA evolutionary algorithm, while in definition of algo (the GA algorithm), the mutation and crossover methods are rewriten to process a vector deconstructed from MLP – Meimei_Huang Dec 15 '22 at 00:20
  • I don't know if when trying to use DE it requires me to use box constraint or other contraints, how to define the vectors of constraints for a model; though I can get vector from flatting a model, how many elements of a constraint vector, what is the bound for each element? sorry, I am a beginner of such things – Meimei_Huang Dec 15 '22 at 00:24
  • but to put it simply, what problem I have now is when I tried to use similar way to replace algo by DE algorithm I defined from what is provided in that package, I can't, it reported method no match. It seems I don't need to give parameters of crossover and mutation to DE constructor, but it does process successully the difinition of DE, and I just replace the algo by it in that method. – Meimei_Huang Dec 15 '22 at 00:30