0

I am trying to build a neural network using keras functional API and to train the network I am using keras tuner. The model consists of some embedding layers and then some dense layers:

import pandas as pd
import numpy as np
from keras.models import Sequential
from keras.layers import Input, Embedding, Dense, Flatten
from tensorflow import keras
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.utils import plot_model
import matplotlib.pyplot as plt
from kerastuner.tuners import RandomSearch, BayesianOptimization

def build_model(hp):
  model = keras.Sequential()
  activation = hp.Choice('activation',['relu','tanh','linear'])

  optimizer = hp.Choice('optimizer', ['adam', 'sgd', 'rmsprop'])
  in_layers = list()
  em_layers = list()
  for i in range(len(X_train_enc)):
    # calculate the number of unique inputs
    n_labels = len(np.unique(X_train_enc[i]))
    # define input layer
    in_layer = Input(shape=(1,))
    # define embedding layer
    em_layer = Embedding(n_labels, round(n_labels/2))(in_layer)
    # store layers
    in_layers.append(in_layer)
    em_layers.append(em_layer)

  merge = keras.layers.concatenate(em_layers)
  x = Flatten()(merge)
  
  for i in range(hp.Int('num_layers', 1, 6)):
    units = hp.Int(
          'units_' + str(i),
          min_value=8,
          max_value=128,
          step=16
    )
    x = Dense(units, activation=activation)(x)
    drop_rate = hp.Choice('drop_rate_' + str(i),
                            [
                              0.0, 0.1, 0.2, 0.3, 0.4,
                              0.5, 0.6, 0.7, 0.8, 0.9
                            ])
    x = keras.layers.Dropout(rate=drop_rate)(x)


  output = Dense(1, activation='linear')(x)
  model = keras.models.Model(inputs=in_layers, outputs=output)

  model.compile(
      optimizer=optimizer,
      loss=keras.losses.MeanSquaredError(reduction="auto", name="mean_squared_error"), 
      metrics=['accuracy']
  )
  return model

To use the tuner following code is executed:

tuner = BayesianOptimization(
    build_model,
    objective='accuracy',
    max_trials=25,
    executions_per_trial=5,
    directory='drive/MyDrive/Master/train_model/nn_first_reg',
    project_name='nn_bayes_first_reg',
    seed=10)


tuner.search(X_train_enc,y_train)
tuner.results_summary()
best_hyperparameters = tuner.get_best_hyperparameters(1)[0]
model = tuner.hypermodel.build(best_hyperparameters)
history = model.fit(X_train_enc, y_train, epochs=50, validation_split=0.2)
val_acc_per_epoch = history.history['val_accuracy']
best_epoch = val_acc_per_epoch.index(max(val_acc_per_epoch)) + 1
print('Best epoch: %d' % (best_epoch,))

The results from the tuner is used to build the model.

_, accuracy = model.evaluate(X_test_enc, y_test)
print('Accuracy: %.2f' % (accuracy*100))
print(model.summary())
from tensorflow.keras.utils import plot_model
plot_model(model, to_file='drive/MyDrive/Master/plots/results/before_game/nn_first.png')

The output from model.summary() and plot_model are empty. The output looks like this:

Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Total params: 137,329
Trainable params: 137,329
Non-trainable params: 0
__________________________________________________________________________________________________
None

Don't know if this is connected, but the code runs perfectly fine, I just plot the network to see how it is built. I do however get an error when I am trying to save my model:

model.save('drive/MyDrive/Master/SavedModels/nn_first_before.csv')

and the error is:

KeyError: 'input_269_ib-0'

I don't know if these are related.

egginald
  • 11
  • 3

1 Answers1

2

You are mixing imports from the tf.keras and keras libraries, which are not the same libraries and are incompatible, producing the strange problems you see here. Note your imports:

from keras.models import Sequential
from keras.layers import Input, Embedding, Dense, Flatten
from tensorflow import keras
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.utils import plot_model

Here you use some layers from the keras library, and other layers from the tensorflow.keras library, which will not work. Only use imports from one of these libraries.

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140