How to solve AttributeError: module 'tensorflow._api.v2.distribute' has no attribute 'TPUStrategy'

Question

I am working with Keras and Tensorflow in order to create a predictor model. I have only CPU device and I can't execute my code. In the code only use Keras and Kerastuner to search hyperparameters. This is the error trace:

File "file.py", line 537, in <Module>
    params,tuner = search_model(X_train,y_train,trials=t,executions=e)
  File "file.py", line 503, in search_model
    verbose = 0
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/engine/base_tuner.py", line 131, in search
    self.run_trial(trial, *fit_args, **fit_kwargs)
  File "file.py", line 476, in run_trial
    super(MyTuner, self).run_trial(trial, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/engine/multi_execution_tuner.py", line 78,
    trial.trial_id, self._reported_step),
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/engine/tuner.py", line 317, in _get_checkp
    if (isinstance(self.distribution_strategy, tf.distribute.TPUStrategy) and
AttributeError: module 'tensorflow._api.v2.distribute' has no attribute 'TPUStrategy'

I have tried things like:

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

or:

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

or:

with tf.device("/cpu:0"):

Nothing works. This is my info device:

>>> from tensorflow.python.client import device_lib
>>> print(device_lib.list_local_devices())
2021-01-04 20:10:34.173749: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2021-01-04 20:10:34.217597: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2100000000 Hz
2021-01-04 20:10:34.225175: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3c8c000b20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-01-04 20:10:34.225243: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-01-04 20:10:34.229506: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-01-04 20:10:34.229542: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2021-01-04 20:10:34.229584: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (estio108): /proc/driver/nvidia/version does not exist
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 5882703391360358162
, name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 18109344296496597660
physical_device_desc: "device: XLA_CPU device"
]

Thank you so much for your help.

What version of `tensorflow` you are using? – Lukasz Tracewski Jan 04 '21 at 20:02 — Lukasz Tracewski, Jan 04 '21 at 20:02
Thanks - did my suggestion work? – Lukasz Tracewski Jan 06 '21 at 11:03 — Lukasz Tracewski, Jan 06 '21 at 11:03

score 1 · Accepted Answer · answered Jan 05 '21 at 09:53

1

You have to update Tensorflow to version 2.3 or greater - that's where the TPUStrategy was added. Here's is your clue to this:

AttributeError: module 'tensorflow._api.v2.distribute' has no attribute 'TPUStrategy'

In this case it'd hardly matter anyway, as you will be using CPU.

answered Jan 05 '21 at 09:53

Lukasz Tracewski

10,794
3
34
53

This suggestion did not work for me. Using version 2.3.1 and got the same error. – Darien Schettler Jan 16 '21 at 17:11
@DarienSchettler It is more likely that you are using a different version than you think. If for sure it's the right one, then it is a bug and the best place to get help would be the project page on Github. – Lukasz Tracewski Jan 16 '21 at 18:23

score 0 · Answer 2 · answered Mar 19 '21 at 18:14

I had the same issue.

The error mentions line 17 in tuner.py and the line

if (isinstance(self.distribution_strategy, tf.distribute.TPUStrategy) and

If you go to the source code locally on your machine, you will see the above line of code actually starts at line 325 as shown below.

Since I was not using TPUs, I decided to comment out the problematic if statement (i.e. lines 325-229 in the pic) looking for TPUStrategy.

It worked!

Useful info:

I tested with TensorFlow 2.2
So far I've only tested a simple model (e.g. Conv2D, Flatten, & Dense layers), so other issues could arise. However, if you're not using a TPU, it seems like that if statement would evaluate to False anyways. But you might get a different error if keras-tuner is using parts of a TensorFlow version which are not compatible with the TensorFlow version you sue.

How to solve AttributeError: module 'tensorflow._api.v2.distribute' has no attribute 'TPUStrategy'

2 Answers2