0

I'm having trouble making an MLP in MxNet learn. It tends to output fairly constant values, only occasionally outputting anything different. I'm using the Pima Indians dataset to do binary classification, but no matter what I do (normalisation, scaling, changing activations, objective functions, number of neurons, batch size, epochs) it wouldn't produce anything useful.

The same MLP in Keras works fine.

Here's the MxNet code:

batch_size=10
train_iter=mx.io.NDArrayIter(mx.nd.array(df_train), mx.nd.array(y_train), 
batch_size, shuffle=True)
val_iter=mx.io.NDArrayIter(mx.nd.array(df_test), mx.nd.array(y_test), batch_size)

data=mx.sym.var('data')

fc1 = mx.sym.FullyConnected(data=data, num_hidden=12)
act1 = mx.sym.Activation(data=fc1, act_type='relu')

fc2 = mx.sym.FullyConnected(data=act1, num_hidden=8)
act2 = mx.sym.Activation(data=fc2, act_type='relu')

fcfinal = mx.sym.FullyConnected(data=act2, num_hidden=2)
mlp = mx.sym.SoftmaxOutput(data=fcfinal, name='softmax')

mlp_model = mx.mod.Module(symbol=mlp, context=mx.cpu())
mlp_model.fit(train_iter,
          eval_data=val_iter,
          optimizer='sgd',
          eval_metric='ce',
          num_epoch=150)

And the same MLP in Keras:

model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(df_train_res, y_train_res)
Sasquatch Man
  • 73
  • 6

1 Answers1

1

I would recommend that you initialize your parameters before you start training. Having all parameters start at zero is not ideal.

You could add the following as a parameter to your model.fit()

initializer=mx.init.Xavier(rnd_type='gaussian')

See here for more discussion https://mxnet.incubator.apache.org/api/python/optimization.html

Simon Corston-Oliver
  • 226
  • 1
  • 6