Questions tagged [tf.data.dataset]
145 questions
0
votes
1 answer
Keras predictions on tf.data dataset doesn't stop
I have run into a problem with using a generator to do predictions for my model.
According to the documentation if steps = None predict should run through the whole dataset.
But instead, it goes passed it and I starts again. I'm using…
Olli
- 906
- 10
- 25
0
votes
2 answers
Drop bad data from dataset Tensorflow
I have a training pipeline using tf.data. Inside the dataset there is some bad elements, in my case a values of 0. How do i drop these bad data elements based on their value? I want to be able to remove them within the pipeline while training since…
David Bacelj
- 172
- 1
- 6
0
votes
1 answer
Efficient way of reshaping a ndarray using sliding window without to use too much memory
I have to reshape a ndarray of [17205, 21] as [17011, 96, 100, 21] by applying two sliding windows to it.
In: arr
Out: [[ 8. 0. 0. -0. 0. 0. 8. 8. 0. 0. 0. 0. 8. 7. 6. 9. 9. 1.
1. 1. 2.]
[ 8. 0. 0. -0. 0. 0. 8. 8. 0. 0.…
Marlon Teixeira
- 334
- 1
- 14
0
votes
1 answer
How to feed the output of PostgreSQL database from TensorFlow IO to tf.keras Sequential Model
I was trying to learn how to read a SQL dataset into Tensorflow using this tutorial. Until the last step everything works fine!
dataset = tfio.experimental.IODataset.from_sql(
query="SELECT pt08s1, nmhc, c6h6, co FROM AirQualityUCI;",
…
ARAT
- 884
- 1
- 14
- 35
0
votes
2 answers
How to feed images into a CNN
I am new to tensorflow, and am trying to create a convolutional neural network for binary classification that can distinguish the difference between a normal face and the face of someone who is having a stroke.
The images for my dataset are…
Colin Tree
- 1
- 3
0
votes
1 answer
tf.data: function fails tries to convert to numpy array?
I'm trying to build a tf.data pipeline, ultimately to compute skipgrams, but I get an error
NotImplementedError: Cannot convert a symbolic Tensor (cond/Identity:0) to a numpy array.
This error may indicate that you're trying to pass a Tensor to a…
Patrick McCarthy
- 2,478
- 2
- 24
- 40
0
votes
1 answer
Accessing Images and Labels Inside a tf.data.Dataset Object
I'm following along the keras tutorial on image classification. I have created a tf.data.Dataset and specified a single batch using the .take() method:
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
"data",
…
Matt_Haythornthwaite
- 662
- 6
- 14
0
votes
1 answer
Tensorflow: How to set tensor shape after reading image from tfrecord file for data augmentation?
I have a tf.data.Dataset that I read from a tfrecords file like so:
import tensorflow as tf
# given an existing record_file
raw_dataset = tf.data.TFRecordDataset(record_file)
example_description = {
"height": tf.io.FixedLenFeature([],…
Christian Steinmeyer
- 824
- 11
- 23
0
votes
1 answer
Get elements of tuple from tf.data.Dataset
I am building a neural network with two input nodes that are connected to an embedding layer each.
I have created a tf.data.Dataset with a tuple as input for the model.
How can I split the tensors in the tuple to forward the first tensor (scalar) to…
silem
- 21
- 7
0
votes
1 answer
Optimal ways to calculate class weights for large datasets
I wanted to know about optimal ways to calculate class weights for large datasets built with the tf.data API. Consider the official TensorFlow tutorial on handling imbalance datasets -…
S. P
- 368
- 3
- 7
- 20
0
votes
1 answer
Why does training a tensorflow model with a tf.data pipeline yield radically different results than directly feeding with EagerTensors?
I am trying to set up a pipeline to train a model. To get started, I am using 'training wheels'.
I preprocess all of my data into 5 EagerTensors -- 3 for features, 2 for targets.
For the sake of argument, lets call the feature tensors "in_a, in_b,…
Matt Skogmo
- 1
- 1
0
votes
2 answers
tf.data, tf.distribute without GPU
I don't have a GPU on my machine, since most of the performance recommondations on tensorflow mention only GPU, can someone confirm that e.g.
tf.data.prefetch
tf.distribute.mirroredstrategy
tf.distribute.multiworkerstrategy
Will only work with…
Max
- 1
0
votes
1 answer
How to split a tensorflow dataset
I have a set of data stored in csv file, currently I read it out and store it np, and then transfer it into Dataset use below code
def read_data():
with open(fname, "r") as f:
lines = f.read().split("\n")
header =…
lqp718
- 1
- 2
0
votes
1 answer
How to feed CNN with tf.data.Dataset
I'm new with tensorflow. i'm trying to run the convolutional neural network for the binary classifitication between cats and dogs.
The data is structured this way: within a directory called data, there are two subdirectories: test and train. within…
0
votes
1 answer
How can I modifya sequencial data using map or filter or reduce method for tf.data.Dataset objects?
I have a python data generator-
import numpy as np
import tensorflow as tf
vocab_size = 5
def create_generator():
'generates sequences of varying lengths(5 to 7) with random number from 0 to voca_size-1'
count = 0
while count < 5:
…
n0obcoder
- 649
- 8
- 24