0

I ran my model yesterday on Google colab using their GPU and no error was encountered.

However, today I tried to run the same code (unchanged from yesterday) and I get the following error:

UnknownError: Graph execution error:
 Fail to find the dnn implementation.
 [[{{node CudnnRNN}}]]
 [[sequential/bidirectional/forward_lstm/PartitionedCall]] [Op:__inference_train_function_6026]

The following is my model:

model.add(Embedding(vocab_size,embedding_vector_length,input_length=X_train.shape[1]))
model.add(Bidirectional(LSTM(250,return_sequences=True)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(128,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(64,activation='relu'))
model.add(Dense(32,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(16,activation='relu'))
model.add(Dense(num_classes,activation='softmax')) #sigmoid or relu-
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) #if relu, use mse or mae #maybe oversample lower classes 
callbacks = [EarlyStopping(monitor='val_loss', patience=3),ModelCheckpoint('../model/model.h5', save_best_only=True,save_weights_only=False)]
history = model.fit(X_train, y_train, validation_split=0.11, epochs=15, batch_size=32, verbose=1, callbacks=callbacks) #epochs will stop before 15 due to callbacks

Error occurs at this line:

history = model.fit(X_train, y_train, validation_split=0.11, epochs=15, batch_size=32, verbose=1, callbacks=callbacks) #epochs will stop before 15 due to callbacks

I tried solutions from other threads but none of them worked. Funny enough, it doesn't happen when I use CPU.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
  • 1
    This happened to be as well. The code ran fine the day before but today it did not work – Dylan Aug 18 '22 at 05:06
  • 1
    Same to me, worked with TPU, CPU, however since yesterday the GPU had this issue through. A lot guys have reported https://github.com/googlecolab/colabtools/issues – TeeTracker Aug 18 '22 at 06:48

0 Answers0