Questions tagged [tf.data.dataset]
145 questions
0
votes
2 answers
Iterating on Tensorfow Dataset returns always a differently sorted array
Suppose you have a tensorflow dataset that has values and labels. In my case I created it from a time series as:
f = pd.read_csv('MY.csv', index_col=0, parse_dates=True)
#extract the column we are interested in
single_col = df[['Close']]
#Convert…
roschach
- 8,390
- 14
- 74
- 124
0
votes
1 answer
Steps per epoch doesn't behave correctly with tensorflow tf.data.Dataset
I am having a problem using the steps_per_epoch parameter in model.fit. Up to my knowledge, the steps_per_epoch is used to log metrics, like a loss function, after a number of iterations, i.e., after feeding a number of batches to the model.…
I. A
- 2,252
- 26
- 65
0
votes
0 answers
Not Getting values of tensor in function which is called from dataset.map in tensorflow
I am trying to get values of tensor from function which is called by map function. But I am getting this error.
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'args_0' with dtype…
sahil makandar
- 131
- 12
0
votes
0 answers
How to enable eager mode in tf.data
For a project, I am using tf.data.Dataset to write the input pipeline.
The input is an image RGB
The label is a list of 2D coordinates of of objects in the image that used to generate a heatmap
Here is a MWE to reproduce the problem.
def…
LearnToGrow
- 1,656
- 6
- 30
- 53
0
votes
1 answer
Tensorflow Estimator: Samples using weighted distribution (probability)
I want to sample datas using weighted distribution (probability)
The examples are like below:
class distribution:
doc_distribution = {0: 40, 1: 18, 2: 8, 3: 598, ... , 9: 177}
I would to make the batch of dataset by equal probability of…
Gi Yeon Shin
- 357
- 2
- 7
- 19
0
votes
1 answer
tf.keras.backend.function for transforming embeddings inside tf.data.dataset
I am trying to use the output of a neural network to transform data inside tf.data.dataset. Specifically, I am using a Delta-Encoder to manipulate embeddings inside the tf.data pipeline. In so doing, however, I get the following…
Ottpocket
- 77
- 12
0
votes
1 answer
Can you interleave a tf.data dataset from multiple files?
I currently have a dataset that is split into three different npy file types: one containing all the x vars that are floats, one containing all the x vars that are ints, and one containing all the y-labels.
To loop through all the files, I created a…
hyw2
- 1
- 1
0
votes
1 answer
How to add time step dimension to generated batch by tf.data.Dataset API
I want add time step dimension to my batch generation.
Currently I am doing
train_ds = tf.data.Dataset.\
from_tensor_slices((x_train, y_train)).\
shuffle(10000).\
batch(32)
and getting batches of size (32, feature_vector_length)
I want…
Night Walker
- 20,638
- 52
- 151
- 228
0
votes
2 answers
How to access tensor shape inside map function
I need to access image shapes to perform an augmentation pipeline although when accessing through image.shape[0] and image.shape[1] I'm unable to perform the augmentations since it outputs that my tensors have shape None.
Related issues: How to…
Luciano Dourado
- 473
- 4
- 14
-1
votes
1 answer
Min-max normalization when using tf.data.Dataset
I have a tf.Dataset and I want to perform a minmax normalization, in order to have image values in the range [0,1].
I am interested in knowing how to perform normalization on each image, as well as for the whole batch.
@tf.function def…
aDav
- 41
- 8