4

I'm trying to run the example from https://www.tensorflow.org/programmers_guide/reading_data with my custom data that looks like:

example_data.txt

DESC|manner|How did serfdom develop in and then leave Russia ?
ENTY|cremat|What films featured the character Popeye Doyle ?
DESC|manner|How can I find a list of celebrities ' real names ?
ENTY|animal|What fowl grabs the spotlight after the Chinese Year of the Monkey ?
                                      more...

TensorFlow code:

import numpy as np
import tensorflow as tf

filename_queue = tf.train.string_input_producer(["example_data.txt"])
reader = tf.TextLineReader()
key, value = reader.read(filename_queue)

record_defaults = [['hello'], ['hello'], ['hello']]
col1, col2, col3 = tf.decode_csv(
    value, record_defaults=record_defaults, field_delim="|")
features = tf.stack([col1, col2, col3])

print(features)

with tf.Session() as sess:
  # Start populating the filename queue.
  coord = tf.train.Coordinator()
  threads = tf.train.start_queue_runners(coord=coord)

  for i in range(1200):
    # Retrieve a single instance:
    example, label = sess.run([features, col2])

  coord.request_stop()
  coord.join(threads)

This throws an error: Expect 3 fields but have 5 in record 0

but if I change:

record_defaults = [['hello'], ['hello'], ['hello'], ['hello'], ['hello']]
    col1, col2, col3, col4, col5 = tf.decode_csv(
        value, record_defaults=record_defaults, field_delim="|")

This throws an error: Expect 5 fields but have 3 in record 0

What is happening? is this a bug in TensorFlow?

user47376
  • 2,263
  • 3
  • 21
  • 26

1 Answers1

4

When using tf.decode_csv with tf.TextLineReader, the mere presence of a blank line at the end of the file caused a similar error 'Expect 5 fields but have 0 in record 0'.

Similarly, you might want to check out your data file to see if there is any wrong with it, even if it is as trivial as "a blank line".

Mark
  • 90,562
  • 7
  • 108
  • 148
Lujun Weng
  • 101
  • 5