Using Class Weights or Sample Weights for One-Hot Encoded labels with keras models

Question

I want to use class weights or sample weights to balance my data during model training. My dataset is an images dataset where we have 20 classes in total. The dataset is highly imbalanced.

I have created a data loader that loads multiple-images alongside its one class-label to keras model during training. The class-labels are converted to One-Hot Encoded form as we are using softmax layer as model's output layer with 20 units(no.of classes). Attached below is the code for my model, and the data loader I am using for training of model:


def data_loader(basePath, X, y, batch_size=16, classDict=None): 
  while True:
    x_batch = np.zeros((batch_size, 4, 64, 64, 3))     
    y_batch = np.zeros((batch_size, 20))           
    
    for i in range(0, batch_size):
        *images, = X.iloc[i]
      
        for j in range(4):
          x_batch[i,j] = plt.imread(os.path.join(basePath, images[j]))  / 255.

        class_label = y.iloc[i][-1]
        class_id = classDict[class_label]
        
        # # using one-hot encoding for assigning class labels
        y_batch[i, class_id] = 1.0

    yield {'image1': x_batch[:, 0], 
                'image2': x_batch[:, 1], 
                'image3': x_batch[:, 2], 
                'image4': x_batch[:, 3], }, {'class_out': y_batch}

# The Model:

input_1 = Input(shape=(64, 64, 3), name='image1')              
input_2 = Input(shape=(64, 64, 3), name='image2')             
input_3 = Input(shape=(64, 64, 3), name='image3')              
input_4 = Input(shape=(64, 64, 3), name='image4') 
filters = [8, 8, 16, 16] 

x1 = input_1
for i in range(0, len(filters)-1):
  n_filters = filters[i]                                      
  x1 = Conv2D(filters= n_filters, kernel_size= 3, activation='relu')(x1)
  x1 = BatchNormalization()(x1)
  x1 = MaxPool2D(2)(x1)

x2 = input_2
for i in range(0, len(filters)-1):
  n_filters = filters[i]                                       
  x2 = Conv2D(n_filters, 3, activation='relu')(x2)
  x2 = BatchNormalization()(x2)
  x2 = MaxPool2D(2)(x2)

x3 = input_3
for i in range(0, len(filters)-1):
  n_filters = filters[i]                                       
  x3 = Conv2D(n_filters, 3, activation='relu')(x3)
  x3 = BatchNormalization()(x3)
  x3 = MaxPool2D(2)(x3)

x4 = input_4
for i in range(0, len(filters)-1):
  n_filters = filters[i]                                      
  x4 = Conv2D(n_filters, 3, activation='relu')(x4)
  x4 = BatchNormalization()(x4)
  x4 = MaxPool2D(2)(x4)

x = concatenate([x1, x2, x3, x4])

x = Flatten()(x)
x = Dense(256, activation='relu')(x)
class_out = Dense(20, activation='softmax', name='class_out')(x)  #Dense layer with 20 units for class label

model = tf.keras.models.Model([input_1, input_2, input_3, input_4], class_out)

# Model Training:
batch_size= 32
_ = model.fit(
    data_loader(X_train, y_train, batch_size, classDictTrain),
    epochs=10,
    steps_per_epoch= df_train.shape[0]//batch_size,
    # class_weight = class_weights,
    # sample_weight = sample_weights,
    validation_data = data_loader(X_val, y_val, batch_size, classDictValid),
    validation_steps = df_validation.shape[0]//batch_size)

I am using sklearn compute_class_weight and compute_sample_weigh for getting weights for my model. But I am facing many errors. I have two queries:

Which of the sklearn functions is actually suitable incase of balancing one-hot encoded labels data?
How should I use these functions for my problem where data is being generated/loaded at time of training in batches and hence the labels are available at that time only? Should I use compute_class_weight function with model.fit() or before it?

Any suggestions might help.

Using Class Weights or Sample Weights for One-Hot Encoded labels with keras models

0 Answers0