Reading Images from TFrecord using Dataset API and showing them on Jupyter notebook

Question

I created a tfrecord from a folder of images, now I want to iterate over entries in TFrecord file using Dataset API and show them on Jupyter notebook. However I'm facing problems with reading tfrecord file.

Code I used to create TFRecord

def _bytes_feature(value):
    return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

def _int64_feature(value):
    return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

def generate_tfr(image_list):
    with tf.python_io.TFRecordWriter(output_path) as writer:
        for image in images:
            image_bytes = open(image,'rb').read()
            image_array = imread(image)
            image_shape = image_array.shape
            image_x, image_y, image_z = image_shape[0],image_shape[1], image_shape[2]

            data = {

              'image/bytes':_bytes_feature(image_bytes),
              'image/x':_int64_feature(image_x),
              'image/y':_int64_feature(image_y),
              'image/z':_int64_feature(image_z)
            }

            features = tf.train.Features(feature=data)
            example = tf.train.Example(features=features)
            serialized = example.SerializeToString()
            writer.write(serialized)

Code to read TFRecord

#This code is incomplete and has many flaws. 
#Please give some suggestions in correcting this code if you can

def parse(serialized):
    features = \
    {
        'image/bytes': tf.FixedLenFeature([], tf.string),
        'image/x': tf.FixedLenFeature([], tf.int64),
        'image/y': tf.FixedLenFeature([], tf.int64),
        'image/z': tf.FixedLenFeature([], tf.int64)
    }

    parsed_example = tf.parse_single_example(serialized=serialized,features=features)
    image = parsed_example['image/bytes']
    image = tf.decode_raw(image,tf.uint8)
    x = parsed_example['image/x'] # breadth
    y = parsed_example['image/y'] # height
    z = parsed_example['image/z'] # depth
    image = tf.cast(image,tf.float32)

    # how can I reshape image tensor here? tf.reshape throwing some weird errors.

    return {'image':image,'x':x,'y':y,'z':z}



dataset = tf.data.TFRecordDataset([output_path])
dataset.map(parse)
iterator = dataset.make_one_shot_iterator()
next_element = iterator.get_next()
epoch = 1

with tf.Session() as sess:
    for _ in range(epoch):
    img = next_element.eval()
    print(img)
    # when I print image, it shows byte code.
    # How can I convert it to numpy array and then show image on my jupyter notebook ?

I've never worked with any of this before and I'm stuck at reading TFRecords. Please answer how to iterate over the contents of TFrecords and show them on Jupyter notebook. Feel free to correct/optimize both pieces of code. That would help me a lot.

score 1 · Answer 1 · answered Jul 17 '18 at 05:30

1

Is this what you may be looking for? I think once u convert to numpy array you can show in jupyter notebook using PIL.Image.

convert tf records to numpy => How can I convert TFRecords into numpy arrays?

show numpy array as image https://gist.github.com/kylemcdonald/2f1b9a255993bf9b2629

answered Jul 17 '18 at 05:30

Mahesh

1,583
13
18

Thanks for the answer Mahesh. The example you posted works fine. However I want to use Dataset APi, it's map function, iterator to generate images. I tried replacing some of the code in your first link with Dataset code and still no luck. – Uchiha Madara Jul 17 '18 at 09:47
Do the image decoders work for you? I generally don't poke around in the dark, but I am also new to dataset api's and image parsing. https://www.tensorflow.org/api_guides/python/image#Encoding_and_Decoding – Mahesh Jul 18 '18 at 06:59
1

Actually I got it work, a silly mistake. In the 2nd code that I posted, I didn't assign the dataset to a variable after parsing. it is `dataset.map(parse)`, but it should be `dataset = dataset.map(parse)`. Then I used the code in your 2nd link to display. – Uchiha Madara Jul 18 '18 at 09:25
1

ah yes..good catch.. btw ive read enough about tf for people to want to switch to keras. – Mahesh Jul 25 '18 at 15:00
1

I switched to pytorch and I'm feeling unstoppable. – Uchiha Madara Jul 25 '18 at 15:07

Reading Images from TFrecord using Dataset API and showing them on Jupyter notebook

1 Answers1