'strided_slice_19' (op: 'StridedSlice') with input shapes

Question

I tried to use keras.preprocessing.image.ImageDataGenerator on TPU,but i get this error from the first epoch.the same code work with jupyter notebook but take hours for trainig.

My MODEL:

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(220))
model.add(Activation('relu'))
model.add(Dropout(0.4))
model.add(Dense(120))
model.add(Activation('softmax'))

Optimizer

opt = tf.train.AdamOptimizer(learning_rate)

model.compile(
      optimizer=opt,
      loss='categorical_crossentropy',
      metrics=['acc'])

Convert Keras to TPU

try:
    device_name = os.environ['COLAB_TPU_ADDR']
    TPU_ADDRESS = 'grpc://' + device_name
    print('Found TPU at: {}'.format(TPU_ADDRESS))

except KeyError:
    print('TPU not found')
tpu_model = tf.contrib.tpu.keras_to_tpu_model(
    model,
    strategy=tf.contrib.tpu.TPUDistributionStrategy(
        tf.contrib.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)))

ImageDataGenerator

train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='categorical')#binary ,categorical

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='categorical')

Model Fit

model_fit=tpu_model.fit_generator(
    train_generator,

    epochs=50,
    steps_per_epoch=60, 

)

I get this error

Epoch 1/50 15/33 [============>.................] - ETA: 8s - loss: 4.7722 - acc: 0.0083INFO:tensorflow:New input shapes; (re-)compiling: mode=train (# of cores 8), [TensorSpec(shape=(0,), dtype=tf.int32, name='core_id_60'), TensorSpec(shape=(0, 128, 128, 3), dtype=tf.float32, name='conv2d_3_input_20'), TensorSpec(shape=(0, 120), dtype=tf.float32, name='activation_13_target_30')] --------------------------------------------------------------------------- InvalidArgumentError Traceback (most recent call last) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs) 1658
try: -> 1659 c_op = c_api.TF_FinishOperation(op_desc) 1660 except errors.InvalidArgumentError as e:

InvalidArgumentError: slice index 0 of dimension 0 out of bounds. for 'strided_slice_19' (op: 'StridedSlice') with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <0>, input[2] = <1>, input[3] = <1>.

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) 17 frames /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs) 1660
except errors.InvalidArgumentError as e: 1661 # Convert to ValueError for backwards compatibility. -> 1662 raise ValueError(str(e)) 1663 1664 return c_op

ValueError: slice index 0 of dimension 0 out of bounds. for 'strided_slice_19' (op: 'StridedSlice') with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <0>, input[2] = <1>, input[3] = <1>.

'strided_slice_19' (op: 'StridedSlice') with input shapes

0 Answers0