Questions tagged [tf.data.dataset]
145 questions
1
vote
0 answers
loading data using tf.from_tensor_slices yields OperatorNotAllowedInGraphError
I am trying to use tf.data pipeline to get finer control over loading image data but I receive the following error which I think is because of usage of list comprehension. My Code looks like this:
def load_files(data_dir: str, val_split=0.2):
…
Jeethesh
- 31
- 5
1
vote
0 answers
Why does tf.data.Dataset.map(function) change the shape of the input image mask when the function being used to map is wrapped with tf.numpy_function?
I'm trying to make a tf.data.Dataset instance for a medical image instance segmentation pipeline, but the output of the masks keep getting expanded with 1 in the first dimension.
def createDataset(df, mode="segment"):
""" Want this to take in a…
Phan Nguyen
- 11
- 1
- 3
1
vote
2 answers
Load custom Data into a tensorflow pipeline
I am trying to implement this code which loads the data from the
official tensorflow dataset to get it to load my data placed on my google drive
dataset, metadata = tfds.load('cycle_gan/horse2zebra',
with_info=True,…
Muhammad Afaq Saeed
- 43
- 7
1
vote
1 answer
How does one deal with categorical labels in TensorFlow?
How does one transform (e.g., one-hot encode, index, bucketize, embed, etc.) labels natively in TensorFlow? tf.feature_column is the preferred way for features, but what about the labels (i.e., targets)? Those too may often need to be transformed…
Tfovid
- 761
- 2
- 8
- 24
1
vote
1 answer
Batch size not passed to tf.keras model: "Error when checking input: expected input1 to have 3 dimensions, but got array with shape (a,b)"
I'm new to tensorflow (v 2.4.1), so this may be very trivial, but I cannot figure it out on my own. I'm passing 2-dimensional (30,1024) tensors to my 2-input tf.keras model through tf.data.Dataset. After batching, the dataset is printed…
Martin
- 11
- 3
1
vote
0 answers
TensorFlow 2.0: Unable to train subclass model with custom fit in graph mode
The code snippet below is a vanila implementation of a TensorFlow model in which I am using subclass model and a custom fit function (implemented through train_step and test_step). The code works fine in the eager execution mode (default mode of…
Milan Jain
- 459
- 7
- 17
1
vote
1 answer
Input shape axis 0 must equal 4, got shape [5] when trying to crop image batch in tensorflow dataset pipeline
I get the following error when I try to crop a batch of images inside a tf.data.Dataset pipeline:
InvalidArgumentError: Input shape axis 0 must equal 4, got shape [5]
[[{{node crop_to_bounding_box/unstack}}]] [Op:IteratorGetNext]
def…
evolved
- 1,850
- 19
- 40
1
vote
0 answers
Getting 'ValueError: Could not find matching function to call loaded from the SavedModel.' on training a model
I'm using Tensorflow 2 and LaBSE pre-trained model on tf-hub (not much familiar with both). (https://tfhub.dev/google/LaBSE/2). I am trying to train a multi class classifier with a custom text data set. I am also following this example on BERT…
Vinura
- 51
- 9
1
vote
1 answer
Tensorflow 1.12 problem with tf.data.Dataset.map() - loading images files using tf.data API
I am trying to load image dataset on to GPU using tf.data API since they offer optimized perfomance. But unfortunately using tf.data.Dataset.map() function doesnt return a dataset compatible with model.fit() or model.fit_generator().
Assume the…
Jeethesh
- 31
- 5
1
vote
1 answer
Accessing tensor numpy array using `dataset.map()` in tensorflow
I am trying to access the numpy array from a tensor object that is processed with https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map.
I get the error: AttributeError: 'Tensor' object has no attribute 'numpy'
When I try to access the…
vbfh
- 115
- 1
- 8
1
vote
1 answer
Slicing in tf.data causes "iterating over `tf.Tensor` is not allowed in Graph execution" error
I have a dataset created as follows where image_train_path is a list of image file paths,
eg. [b'/content/drive/My Drive/data/folder1/im1.png', b'/content/drive/My Drive/data/folder2/im6.png',...]. I need to extract the folder paths eg,…
momo
- 1,052
- 5
- 16
- 34
1
vote
1 answer
How to create a tf.data pipeline with multiple .npy files
I have looked into other issues on this problem but could not find the exact answer, so trying from scratch:
The problem
I have multiple .npy files (X_train files) each an array of shape (n, 99, 2) - only the first dimension differs, while the…
codeless
- 49
- 5
1
vote
1 answer
Tensorflow 2.3 pipeline load all the data to the RAM
I created pipeline using tf.data API, for reading data set of images. I have a big dataset with high resolution. However, each time trying to reading all the dataset, the computer crash because the code using all the RAM. I tested the code with…
Zaher88abd
- 448
- 7
- 20
1
vote
1 answer
Changing label_mode of tf.keras.preprocessing.image_dataset_from_directory after setting it once
I'm using this code to load images that I have to pass to a Convolutional variational autoenocder:
import tensorflow as tf
train = tf.keras.preprocessing.image_dataset_from_directory(
data_dir + 'Train/', label_mode=None,
…
Imanpal Singh
- 1,105
- 1
- 12
- 22
1
vote
1 answer
Sequential model with tensorflow dataset
I tried to understand how to use tensorflows Datasets for a simple regression model, instead of feeding it with a separate np.array for training input and output.
Here a simple standalone example:
import tensorflow as tf
import numpy as np
# create…
Max86
- 55
- 1
- 4