Questions tagged [tf.data.dataset]
145 questions
0
votes
0 answers
Best way to load frames of video with tensorflow
I was working on a video classification problem. the data is given as frames per video so I created a generator that takes an array of videos with dim = (batch_size, #frames, IMG_SIZE, IMG_SIZE, 3) with keras.utils.Sequence
I was working on Kaggle…
yahia zakaria
- 1
- 1
0
votes
1 answer
Make .zip of discreate dataset using tensorflow
I have two dummy image dataset with three elements in the first and 6 elements in the second dataset.
like 1st dataset images name = [1.png, 2.png, 3.png]
2nd dataset images name = [1_1.png, 1_2.png, 2_1.png, 2_2.png, 3_1.png,…
Han
- 5
- 3
0
votes
1 answer
How to apply a function to convert the paths to arrays using cv2 in tensorflow data pipeline?
Any help will be highly appreciated
I'm trying to load two lists containing image paths and their corresponding labels. Something like this:
p0 = ['a','b',....] #paths to images .tif format
p1 = [1,2,3,......] #paths to images .tif format
labels =…
Mainak Sen
- 63
- 6
0
votes
1 answer
Fashion Mnist Tensorflow Data Shape Incompatibility
I know that there are similar questions. Although I have checked them, I did not solve my problem.
I tried to implement mini-batching on fashion-Mnist dataset. Therefore I converted the dataset from np.array to tensor with…
Berkay Kepekçi
- 1
- 4
0
votes
0 answers
Remove mean image from tf.data.Dataset of complex-valued images
I have a tf.data.Dataset called train_dataset such that -
a, b = next(iter(train_dataset))
print(a.shape, b.shape)
output - (100, 128, 128, 2) (100, 128, 128, 1)
That is, my batch size is 100, input images are 128*128 sized complex-valued images…
psj
- 356
- 3
- 18
0
votes
2 answers
How to use list of sparse tensors in tf.data.Dataset?
I'm trying to build a model which takes list of sparse tensors as input. (list length is equal to batch size)
The reason I use sparse tensor is that I have to pass adjacency matrix to my GNN model and it is very sparse. (~99%)
I'm familiar with…
zzaebok
- 31
- 7
0
votes
1 answer
How to convert a large string data array from file to np.array with float data type
My task is to direct a potentially large set of elements for training a neural network. I am trying to use tf.data.experimental.CsvDataset and tf.data.experimental.make_csv_dataset but I keep getting stuck.
My dataset is a text file containing…
Andrei
- 1
- 1
0
votes
0 answers
Wrapping image_data_generator.flow_from_dataframe in tf.data pipeline.. what step should I take?
train_generator=datagen.flow_from_dataframe(dataframe=train_df, #directory=data_path,
x_col="Path", y_col="feature_string", seed = 42, classes = chexpert_targets,
…
Beckham
- 13
- 4
0
votes
1 answer
Difference between tf.data.TextLineDataset and tf.data.experimental.make_csv_dataset
Open a Google colab notebook and run below statements
#
import tensorflow as tf
import pathlib
import os
dataset = tf.data.TextLineDataset('/content/sample_data/california_housing_test.csv')
dataset ## output is
sakeesh
- 919
- 1
- 10
- 24
0
votes
1 answer
Passing tensors as an argument to a function
i am trying to normalize a tf.data.Dataset as seen below:
def normalization(image):
print(image['label'])
return 1
z = val.map(normalization)
the val dataset is like this :
Epitheoritis 32
- 366
- 5
- 13
0
votes
1 answer
AttributeError: 'Tensor' object has no attribute 'numpy' eager execution is enabled using version 2.4.1
I've been trying to convert a generator I built to a tf.data.dataset.
I've come far and now I have something simple like this
def parse_image(filename):
file = tf.io.read_file(filename) # this will work only with filename as tensor
image =…
Olli
- 906
- 10
- 25
0
votes
0 answers
How to correctly implement tf.data from generator with multiple inputs?
I have created a custom generator for a siamese network which feeds the network triplets of images:
def __getitem__(self, index):
...preprocessing of anchors positive and negative matches
return anchors, positives, negatives
What is the…
Olli
- 906
- 10
- 25
0
votes
0 answers
tf.data input pipeline, dynamic shaped tensors and slicing: NotImplementedError: Cannot convert a symbolic Tensor (args_0:0) to a numpy array
I am trying to write an efficient data input using tensorflow and tf.data.
I want to replicate the functionality of PIL.Image.Image.crop, where one can pass negative bounding box values such that the crop is expanded with zeros.
For example, if I…
benjs
- 15
- 5
0
votes
0 answers
Tensorflow "model.evaluate()" giving different results each time is run on same dataset
I am having different results when I run model.evaluate in Tensorflow more than once in the same validation set.
The model includes data augmentation layers, EfficientNetB0 baseline, and a GlobalAveragePooling layer (see below). I am loading the…
Angel Luis
- 1
- 1
0
votes
1 answer
tf.data.Dataset: Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized
I needed to convert my data generator based on Sequence to tf.data.Dataset format. For that purpose, I used the from_generator function to create repeating BatchedDataset for all my train, validation and testing data.
dataset =…
vcucu
- 184
- 3
- 12