I am training a CNN model as follows:
train_ds = image_dataset_from_directory(
self.data_dir,
validation_split=self.val,
subset="training",
seed=1,
image_size=(self.height, self.width),
batch_size=BATCH_SIZE)
val_ds = image_dataset_from_directory(
self.data_dir,
validation_split=self.val,
subset="validation",
seed=1,
image_size=(self.height, self.width),
batch_size=BATCH_SIZE)
gpus = tf.config.list_logical_devices('GPU')
strategy = tf.distribute.MirroredStrategy(gpus)
with strategy.scope():
model = Sequential([
Conv2D(32, 3, padding='same'),
PReLU(),
MaxPooling2D(),
Flatten(),
Dense(64),
PReLU(),
Dense(num_classes)])
optimizer = Adam()
model.compile(optimizer=optimizer,
loss=SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
history = model.fit(
train_ds,
validation_data=val_ds,
epochs=100,
)
However, I get the following error, and the script terminates:
W tensorflow/core/grappler/optimizers/data/auto_shard.cc:786] AUTO sharding policy will apply DATA sharding policy as it failed to apply FILE sharding policy because of the following reason: Found an unshardable source dataset: name: "TensorSliceDataset/_1"
op: "TensorSliceDataset"
I found this question, but the answer works for tf.data.Datasets
. I wonder if there are any solutions for image_dataset_from_directory()? and this answer proposes to use from_tensor_slices()
, but I don't know how to apply it to a multiclass directory. Thanks in advance for your help.