OutOfRangeError: RandomShuffleQueue '_7_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 200, current size 0)

Question

really thank you to read my problem

My dataset is about 9779 images(.dcm). There are two labels, One is 1, who has 5000 training images, and the other is 0, who has 4779 images.

I use TFrecords to merge and build a dataset. Then feed it to a CNN model.

writer = tf.python_io.TFRecordWriter("train.tfrecords")
    for idx, img_path in enumerate(all_images):#all_images is a list containing all path of all images
        img = dm.read_file(img_path)

        pixel_bytes = img.PixelData 
        img_raw = pixel_bytes
        if idx < len_all_cancer_images:
            example = tf.train.Example(features=tf.train.Features(feature={
            "label": tf.train.Feature(int64_list = tf.train.Int64List(value = [1])),
            'img_raw': tf.train.Feature(bytes_list = tf.train.BytesList(value = [img_raw]))}))
            writer.write(example.SerializeToString())

        else:
            example = tf.train.Example(features=tf.train.Features(feature={
            "label": tf.train.Feature(int64_list = tf.train.Int64List(value=[0])),
            'img_raw': tf.train.Feature(bytes_list = tf.train.BytesList(value = [img_raw]))}))
            writer.write(example.SerializeToString())

    writer.close()

Then I use TFRecordReader to read it

    filename = '/Users/wuzhenglin/Python_nice/SAL_LUNG/train.tfrecords'
    filename_queue = tf.train.string_input_producer([filename])

    reader = tf.TFRecordReader()
    _, serialized_example = reader.read(filename_queue)   
    features = tf.parse_single_example(serialized_example,
                                       features={
                                           'label': tf.FixedLenFeature([], tf.int64),
                                           'img_raw' : tf.FixedLenFeature([], tf.string),
                                       })

    print features['img_raw']
    print features['label']



    img = tf.decode_raw(features['img_raw'], tf.uint8)
    img = tf.reshape(img, [512, 512, 1])
    img = tf.cast(img, tf.float32) * (1. / 255)
    label = tf.cast(features['label'], tf.int32)

Then when I want to read my data

img_batch, label_batch = tf.train.shuffle_batch([img, label],
                                                batch_size = 200, capacity = 9779,
                                                min_after_dequeue = 2000)
init = tf.global_variables_initializer()

    with tf.Session() as sess:
        sess.run(init)

        coord = tf.train.Coordinator()
        threads = tf.train.start_queue_runners(coord = coord)

        for i in xrange(1):
            a, b = sess.run([img_batch, label_batch])
            a_ = a[0]
            b_ = b[0]
            img_1 = tf.reshape(a_[0, :, :, :], [512, 512])
            print img_1.shape
            print b_.shape
            print b_
            print '********************'
            plt.imshow(sess.run(img_1), cmap='gray')

I want to pick some data and print them, but error...

First:

InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 524288 values, but the requested shape has 262144
     [[Node: Reshape = Reshape[T=DT_UINT8, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](DecodeRaw, Reshape/shape)]]

Second:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 880, in runfile
    execfile(filename, namespace)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
    builtins.execfile(filename, *where)
  File "/Users/wuzhenglin/Python_nice/SAL_LUNG/test.py", line 96, in <module>
    read_and_decode()
  File "/Users/wuzhenglin/Python_nice/SAL_LUNG/test.py", line 82, in read_and_decode
    a, b = sess.run([img_batch, label_batch])
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
    feed_dict_string, options, run_metadata)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_7_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 200, current size 0)
     [[Node: shuffle_batch_1 = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch_1/random_shuffle_queue, shuffle_batch_1/n)]]

Caused by op u'shuffle_batch_1', defined at:
  File "<stdin>", line 1, in <module>
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 880, in runfile
    execfile(filename, namespace)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
    builtins.execfile(filename, *where)
  File "/Users/wuzhenglin/Python_nice/SAL_LUNG/test.py", line 96, in <module>
read_and_decode()
  File "/Users/wuzhenglin/Python_nice/SAL_LUNG/test.py", line 71, in read_and_decode
min_after_dequeue = 2000)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1165, in shuffle_batch
name=name)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 739, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1310, in _queue_dequeue_many_v2
    timeout_ms=timeout_ms, name=name)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/Users/wuzhenglin/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in __init__
    self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_7_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 200, current size 0)
     [[Node: shuffle_batch_1 = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch_1/random_shuffle_queue, shuffle_batch_1/n)]]

And some useful information, the image.dcm is 512*512, RGB

Really really thank you :-)

Ishant Mrinal · Answer 1 · 2017-08-04T09:00:12.613

0

1 error InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 524288 values, but the requested shape has 262144 suggests that your input size for the reshape operation is [512, 512, 2]

# this line causes the error,, you have to confirm that your image size
img = tf.reshape(img, [512, 512, 1])
# should be
img = tf.reshape(img, [512, 512, 2])
# another line with mkstake; here you dont need to use tensorflow reshape; you can use numpy reshape since a[0] is python object
# img_1 = tf.reshape(a_[0, :, :, :], [512, 512])
img_1 = np.reshape(a_[0, :, :, :], [512, 512, 2])

2nd error comes up due to the first error. SO make sure you are not making any mistakes with thew input image size.

edited Aug 04 '17 at 09:00

answered Aug 04 '17 at 08:16

Ishant Mrinal

4,898
3
29
47

It still does not work, InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 524288 values, but the requested shape has 262144 [[Node: Reshape = Reshape[T=DT_UINT8, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](DecodeRaw, Reshape/shape)]] – Tozz Aug 04 '17 at 08:29
see the edits, I meant your input image size is not 512, 512, 1; its somehow 512, 512, 2 – Ishant Mrinal Aug 04 '17 at 08:34
But how do I change the reshape value? – Tozz Aug 04 '17 at 08:41
`img = tf.reshape(img, [512, 512, 2])` – Ishant Mrinal Aug 04 '17 at 08:51
Yes, but I do not know why it is the same error no matter how I change the reshape thing. It still has this problem: INFO:tensorflow:Error reported to Coordinator: Input to reshape is a tensor with 524288 values, but the requested shape has 262144 – Tozz Aug 04 '17 at 09:03

OutOfRangeError: RandomShuffleQueue '_7_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 200, current size 0)

1 Answers1