RuntimeError:Mixing different tf.distribute.Strategy objects

Question

Hello！I have encountered some problems when compiling the model using TPU.Some part of codes as follows:

resolver = tf.contrib.cluster_resolver.TPUClusterResolver(TF_MASTER)

tf.contrib.distribute.initialize_tpu_system(resolver)

strategy = tf.contrib.distribute.TPUStrategy(resolver)

with strategy.scope():

  model = create_model()

  model.compile(optimizer=tf.keras.optimizers.Adadelta(),loss='categorical_crossentropy',metrics='accuracy'])

And I got RuntimeError:enter image description here

Can you help me?

Answer 1

I solved my problem by various chaos trying.You can restart your program or comment the code:

resolver = tf.contrib.cluster_resolver.TPUClusterResolver
tf.contrib.distribute.initialize_tpu_system(resolver)
strategy = tf.contrib.distribute.TPUStrategy(resolver)

with strategy.scope():
  model = create_model()
  model.compile()

to avoid the problem

Answer 2

Same problem. It seems the version of TensorFlow as default is 1.x. I change my code into: (comment 3 lines and add other lines)

try:
  # %tensorflow_version only exists in Colab.
  %tensorflow_version 2.x
except Exception:
  pass

# resolver = tf.contrib.cluster_resolver.TPUClusterResolver('grpc://' + os.environ['COLAB_TPU_ADDR'])
# tf.contrib.distribute.initialize_tpu_system(resolver)
# strategy = tf.contrib.distribute.TPUStrategy(resolver)
resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.config.experimental_connect_to_cluster(resolver)
tf.tpu.experimental.initialize_tpu_system(resolver)
strategy = tf.distribute.experimental.TPUStrategy(resolver)

It solved.