3

For Object Detection via the Tensorflow API using the model_main.py, when I use i.e. random_horizontal_flip in the data_augmentation_options in the train_config of my pipeline.config, are my bounding boxes also affected? This is very important, as otherwise these options are not applicable. This is the same question, but it was not properly answered.

Jay Carraway
  • 33
  • 1
  • 4

1 Answers1

5

Yes, the bounding boxes are affected in the same way. Specifically for random_horizontal_flip, you can verify it by looking at the function, which also receives boxes. Flipping the bounding boxes is performed here. Note not all augmentation options need bounding box altering, but those who do - alter the bounding box accordingly.

netanel-sam
  • 1,862
  • 9
  • 14
  • Thank you for your answer @netanel-sam. I could also find the corresponding function in the preprocessor.py, but I was not able to find the part where this information is passed into the function `random_horizontal_flip`. Do you know where exactly this happens? – Jay Carraway Nov 20 '18 at 13:17
  • 1
    It happens here: https://github.com/tensorflow/models/blob/master/research/object_detection/core/preprocessor.py#L3206. `func` is a preprocessing function (iterated over all `preprocess_options`), `args` is the arguments to the preprocessing function such as image, bounding boxes, masks, etc. `params` is parameters to a preprocessing function, e.g. scaling ratio, cropping parameters etc. In case of `random_horizontal_flip` there aren't interesting ones (keypoint permutation). – netanel-sam Nov 21 '18 at 09:00
  • Okay, and where are these `preprocess_options` defined? I can't even find the place where the preprocessor.py is called at all. I have a really hard time navigating through all these connected scripts. I did a 'usage search' with my IDE and it couldn't find any usage of the preprocessor.py as well as the `preprocess` function. I could follow the path of the pipeline.config in the model_main.py and where it gets processed into the `train_and_eval_dict` and then via the `model_lib.create_train_and_eval_specs` into `train_spec` where I suppose the information for the data augmentation is. – Jay Carraway Nov 23 '18 at 09:39
  • But that's pretty much where it ends. `train_spec` is then fed into the `tf.estimator.train_and_evaluate`but I cannot find anything about the preprocessing there. – Jay Carraway Nov 23 '18 at 09:40
  • 1
    It is indeed hard to navigate. You should delve in inputs.py. The function `augment_input_data` receives the tensor which contains the data (images, labels, etc), and the augmentation options, and then applies them in https://github.com/tensorflow/models/blob/master/research/object_detection/inputs.py#L290. This function is called by `transform_and_pad_input_data_fn` which reads the data augmentation options right from the train config file, and creates `data_augmentation_fn` which is then used as part of the all transformations to the input data. – netanel-sam Nov 26 '18 at 09:44