create_coco_tf_record.py from object detection API not creating tfrecord file

Question

I am trying to create tfrecord (.record) files out of coco-format (.json). Therefore I am using this slightly modified version of the create_coco_tf_record.py script from the object detection api github repository.
I am running my notebook on colab. These are the few lines of code:

#Mount Google Drive.
from google.colab import drive
drive.mount('/content/gdrive') <br>

!pip install -U --pre tensorflow=="2.2.0"

#Download TensorFlow Model Garden.
import os
import pathlib
#cd into the TensorFlow directory in your Google Drive
%cd '/content/gdrive/My Drive/TensorFlow'
# Clone the tensorflow models repository if it doesn't already exist
if "models" in pathlib.Path.cwd().parts:
  while "models" in pathlib.Path.cwd().parts:
    os.chdir('..')
elif not pathlib.Path('models').exists():
  !git clone --depth 1 https://github.com/tensorflow/models

# Install the Object Detection API
%%bash
cd '/content/gdrive/My Drive/TensorFlow/models/research/'
protoc object_detection/protos/*.proto --python_out=.
cp object_detection/packages/tf2/setup.py .
python -m pip install .

#run model builder test
!python '/content/gdrive/My Drive/TensorFlow/models/research/object_detection/builders/model_builder_tf2_test.py'

!wget https://raw.githubusercontent.com/TannerGilbert/Tensorflow-Object-Detection-API-train-custom-Mask-R-CNN-model/master/create_coco_tf_record.py

# create tf record
!python create_coco_tf_record.py \
  --logtostderr \
  --train_image_dir='/content/gdrive/My Drive/TensorFlow/workspace/training_demo/images/combined/' \
  --test_image_dir='/content/gdrive/My Drive/TensorFlow/workspace/training_demo/images/combined/' \
  --train_annotations_file='/content/gdrive/My Drive/TensorFlow/workspace/training_demo/images/train_coco.json' \
  --test_annotations_file='/content/gdrive/My Drive/TensorFlow/workspace/training_demo/images/test_coco.json' \
  --output='/content/gdrive/My Drive/TensorFlow/workspace/training_demo/annotations/'

The create_coco_tf_record.py script runs through without any error. This is what it shows:

2020-10-28 08:58:44.931401: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
INFO:tensorflow:Found groundtruth annotations. Building annotations index.
I1028 08:58:46.678869 139783613331328 create_coco_tf_record.py:214] Found groundtruth annotations. Building annotations index.
INFO:tensorflow:0 images are missing annotations.
I1028 08:58:46.679595 139783613331328 create_coco_tf_record.py:227] 0 images are missing annotations.
INFO:tensorflow:On image 0 of 451
I1028 08:58:46.680609 139783613331328 create_coco_tf_record.py:232] On image 0 of 451
INFO:tensorflow:On image 100 of 451
I1028 08:58:51.800869 139783613331328 create_coco_tf_record.py:232] On image 100 of 451
INFO:tensorflow:On image 200 of 451
I1028 08:59:01.762672 139783613331328 create_coco_tf_record.py:232] On image 200 of 451
INFO:tensorflow:On image 300 of 451
I1028 08:59:22.197772 139783613331328 create_coco_tf_record.py:232] On image 300 of 451
INFO:tensorflow:On image 400 of 451
I1028 09:00:17.036898 139783613331328 create_coco_tf_record.py:232] On image 400 of 451
INFO:tensorflow:Finished writing, skipped 0 annotations.
I1028 09:00:32.919734 139783613331328 create_coco_tf_record.py:239] Finished writing, skipped 0 annotations.
INFO:tensorflow:Found groundtruth annotations. Building annotations index.
I1028 09:00:32.932144 139783613331328 create_coco_tf_record.py:214] Found groundtruth annotations. Building annotations index.
INFO:tensorflow:0 images are missing annotations.
I1028 09:00:32.932511 139783613331328 create_coco_tf_record.py:227] 0 images are missing annotations.
INFO:tensorflow:On image 0 of 152
I1028 09:00:32.932658 139783613331328 create_coco_tf_record.py:232] On image 0 of 152
INFO:tensorflow:On image 100 of 152
I1028 09:00:46.510094 139783613331328 create_coco_tf_record.py:232] On image 100 of 152
INFO:tensorflow:Finished writing, skipped 0 annotations.
I1028 09:01:08.650619 139783613331328 create_coco_tf_record.py:239] Finished writing, skipped 0 annotations.

Somehow it does not create any .record files.

Does anyone know what the problem could be here?

Thanks in advance!

Please share a self-contained notebook that reproduces the problem you observe. — Bob Smith, Oct 28 '20 at 15:18
Hey @BobSmith, thank you very much for your answer.This was the solution to my problem! Here is my [colab](https://colab.research.google.com/drive/1gnPBYoY669_K0BH7lrf9v7mRKUi1itd9?usp=sharing) notebook. But I'm still wondering why a self-contained notebook is doing the job and a mounted not. Do you know why? thx — Sungsy, Oct 29 '20 at 07:31

score 0 · Answer 1 · edited Oct 29 '21 at 06:42

You can run this command to generate your .record files in VS code terminal on a windows machine. "Imp. - Do not use 'commas' to include your path as I have mentioned just copy/paste path as such where I have suggested in the comment"-

python create_coco_tf_record.py 
  --train_image_dir 'Path where you have your training dataset' 
  --test_image_dir 'Path where you have your validation dataset' 
  --train_annotations_file 'Path to your training dataset .json file' 
  --test__annotations_file 'Path to your validation dataset .json file' 
  --logtostderr --output_dir 'Path where do you want to generate your .record file'

Also keep in mind and run the code with specific environment requirements needed to run this code.

create_coco_tf_record.py from object detection API not creating tfrecord file

1 Answers1