1

I am using TF-slim to train my own dataset. According to the tutorial, in the finetune_inception_v1_on_flowers.sh, I set the checkpoint path as ./inception_resnet_v2.ckpt, and get lots of new checkpoints in another file folder, and the latest files are model.ckpt-332331.data-00000-of-00001,model.ckpt-332331.index, model.ckpt-332331.meta.

now I want to change the learning rate to fine turing the latest check point, but I don't know how to load them in the finetune_inception_v1_on_flowers.sh because there are 3 files instead of one.

firstly, I tried to copy the  model.ckpt-332331.data-00000-of-00001 and set the  checkpoint path as ./model.ckpt-332331.data-00000-of-00001 ,but there is the error:

NotFoundError (see above for traceback): Key InceptionResnetV2/AuxLogits/Conv2d_1b_1x1/weights/Adam not found in checkpoint [[Node: save/RestoreV2_4 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_4/tensor_names, save/RestoreV2_4/shape_and_slices)]]

secondly, I removed the files from log directory ,then the error is :

DataLossError (see above for traceback): Unable to open table file /media/re/7A308F7E308F405D/xsj/checkpoints/model.ckpt-332331.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator? [[Node: save/RestoreV2_5 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_5/tensor_names, save/RestoreV2_5/shape_and_slices)]]

Can anyone help me to solve the problem? Thank you very much!

aek8
  • 319
  • 1
  • 8
  • 22
xsj
  • 11
  • 2

2 Answers2

2

As mentioned in https://www.tensorflow.org/guide/saved_model

There is not a physical file called /tmp/model.ckpt. It is the prefix of filenames created for the checkpoint. Users only interact with the prefix instead of physical checkpoint files.

So you just need to set checkpoint path as ./model.ckpt-332331

wangyu
  • 150
  • 7
0

When you do fine tune, just specify the directory ($CHECKPOINT_PATH) not any specific file, it will automatically get the path.

Ashiq Imran
  • 2,077
  • 19
  • 17