3d plot of learning rate, momentum and accuracy

Question

I have a basic neural net that I have trained in Keras. I'm playing around with the effect of the learning rate and momentum term and I would like to plot a nice 3d graph to visualise the effect of learning rate and momentum on the accuracy.

I've managed to successfully plot a trisurf plot using the example code, however whenever I use my own data I run into errors. The examples seem to use numpy arrays of around 1000 values, whereas I only have about 6 different learning rate and momentum values, giving me numpy arrays of sizes 6, 6 and 36. When I try to plot the graph using these values, I get the following error:

RuntimeError: Error in qhull Delaunay triangulation calculation: singular input data (exitcode=2)

I'm not understanding this error message, and why it works on the example data, but not my own. Any suggestions?

My code is as follows:

momentum_terms = np.array([0.00001,0.0001,0.001,0.01, 0.1, 1])
learning_rates = np.array([0.00001,0.0001,0.001,0.01, 0.1, 1])
train_accuracies = np.empty([36])
test_accuracies = np.empty([36])
for learning_rate in learning_rates:
    for momentum in momentum_terms:
        model = Sequential()
        model.add(Dense(18, activation='relu', input_shape = (2,)))
        model.add(Dense(18, activation='relu'))
        model.add(Dense(1, activation='sigmoid'))
        model.summary()

        model.compile(loss='binary_crossentropy',
                      optimizer=SGD(lr = learning_rate, momentum = momentum),
                      metrics=[binary_accuracy])

        history = model.fit(x_train, y_train,
                            batch_size=batch_size,
                            epochs=epochs,
                            verbose=1,
                            validation_data=(x_test, y_test))
        score = model.evaluate(x_test, y_test, verbose=0)
        np.append(train_accuracies, history.history['binary_accuracy'][-1] * 100)
        np.append(test_accuracies, history.history['val_binary_accuracy'][-1] * 100)

x = momentum_terms
y = learning_rates
z = test_accuracies

ax = plt.axes(projection='3d')
ax.plot_trisurf(x, y, z, cmap='viridis', edgecolor='none');
plt.show()

Adnan S · Accepted Answer · 2018-03-16T15:12:35.407

You are not providing enough data to generate a 3d plot (see this related SO question). Instead of passing 6, 6, and 36, you need to pass 36, 36, and 36. Redo your code so that you store each pair of the momentum terms and learning rate terms in your loops with your accuracy.

So you should have:

x = [0.00001, 0.00001, 0.00001, 0.00001, 0.00001, 0.00001, 0.0001, 0.0001, .... ] for 36 values total from learning rate choices

y = [0.00001,0.0001,0.001,0.01, 0.1, 1, 0.00001, 0.0001,0.001,0.01, 0.1, 1, .... ] for 36 values total from momentum choices

z = array of 36 accuracies for each of the combination above

3d plot of learning rate, momentum and accuracy

1 Answers1