0

I am training a CNN model as follows:

train_ds = image_dataset_from_directory(
    self.data_dir,
    validation_split=self.val,
    subset="training",
    seed=1,
    image_size=(self.height, self.width),
    batch_size=BATCH_SIZE)

val_ds = image_dataset_from_directory(
    self.data_dir,
    validation_split=self.val,
    subset="validation",
    seed=1,
    image_size=(self.height, self.width),
    batch_size=BATCH_SIZE)

gpus = tf.config.list_logical_devices('GPU')
strategy = tf.distribute.MirroredStrategy(gpus)
with strategy.scope():
    model = Sequential([
        Conv2D(32, 3, padding='same'),
        PReLU(),
        MaxPooling2D(),
        Flatten(),
        Dense(64),
        PReLU(),
        Dense(num_classes)])
    optimizer = Adam()

    model.compile(optimizer=optimizer,
                  loss=SparseCategoricalCrossentropy(from_logits=True),
                  metrics=['accuracy'])

history = model.fit(
    train_ds,
    validation_data=val_ds,
    epochs=100,
)

However, I get the following error, and the script terminates:

W tensorflow/core/grappler/optimizers/data/auto_shard.cc:786] AUTO sharding policy will apply DATA sharding policy as it failed to apply FILE sharding policy because of the following reason: Found an unshardable source dataset: name: "TensorSliceDataset/_1"
op: "TensorSliceDataset"

I found this question, but the answer works for tf.data.Datasets. I wonder if there are any solutions for image_dataset_from_directory()? and this answer proposes to use from_tensor_slices(), but I don't know how to apply it to a multiclass directory. Thanks in advance for your help.

0 Answers0