2

Can someone please help me with why Tensorflow embedding projector is not working? I am training an autoencoder and am now trying to visualize the latent space. I followed this very useful tutorial: https://github.com/anujshah1003/Tensorboard-own-image-data-image-features-embedding-visualization

I have been rechecking my work, and I can't find any errors. The projector starts and ends up at this screen as shown in the attached image. It just says the points and dimensions are loading, but no points actually get loaded. The image I'm using and the code I have is below. Any pointers greatly appreciated. Thank you very much! I am using Tensorflow 1.9.0 with Keras 2.1.6 and Python 2.7. I was using Tensorboard 1.9.0 but downgraded to 1.5.1 though it didn't do anything.

image_list = load_crops(num_positive,num_negative,h5_file,only_positives)
LOG_DIR = os.getcwd() + '/embedding-logs'


#now get the feature vectors by creating the encoder and running images through
embedding = encoder.predict(image_list)
features = tf.Variable(embedding, name='features')


#obtain the labels and name them
n_classes = 2
num_of_samples = embedding.shape[0]
num_of_samples_each_class = num_of_samples/n_classes

y = np.ones((num_of_samples,), dtype = 'int64')
y[:num_of_samples_each_class] = 0 
y[num_of_samples_each_class:num_of_samples_each_class*2] = 1  
names = ['CD3+','FOXP3+']

#generate metadata file that says which features belong to which label
#metadata allows to assign labels to each point in embedded space.  label will be the name and the number we assign
metadata_file = open(os.path.join(LOG_DIR, 'metadata_2_classes.tsv'), 'a+')
metadata_file.write('Class\tName\n')

k=num_of_samples_each_class 
j=0
for i in range(num_of_samples):
    c = names[y[i]]
    if i%k==0:
        j=j+1
    metadata_file.write('{}\t{}\n'.format(j,c))
metadata_file.close()


#we have to generate sprite image if we want to see the images in the visualization
sprite = images_to_sprite(image_list)
cv2.imwrite(os.path.join(LOG_DIR, 'sprite_2_classes.png'), sprite)

#run session
with tf.Session() as sess:
    img_data = image_list

    saver = tf.train.Saver([features])
    sess.run(features.initializer)
    saver.save(sess, os.path.join(LOG_DIR, 'images_2_classes.ckpt'))

    config = projector.ProjectorConfig()
    embedding = config.embeddings.add()
    embedding.tensor_name = features.name
    # Link this tensor to its metadata file (e.g. labels).
    embedding.metadata_path = os.path.join(LOG_DIR, 'metadata_2_classes.tsv')
    # Comment out if you don't want sprites
    embedding.sprite.image_path = os.path.join(LOG_DIR, 'sprite_2_classes.png')
    embedding.sprite.single_image_dim.extend([img_data.shape[1], img_data.shape[1]])
    # Saves a config file that TensorBoard will read during startup.
    projector.visualize_embeddings(tf.summary.FileWriter(LOG_DIR), config)

What shows up on Tensorboard

Ray
  • 21
  • 3

1 Answers1

1

I figured it out. You have to apply np.squeeze on the output of the encoder. The output of the encoder being that the embedding projector has to be an array of points). It can't plot with that extra one so when it is removed, it works.

Vivek Reddy
  • 92
  • 1
  • 1
  • 9
  • Are you also able to visualize the embedding space also with other architecture like Resnet and in Pytorch? – Alberto Tono Nov 20 '20 at 02:26
  • 1
    @AlbertoTono Yes Pytorch has a Tensorboard API. You can use Summary Writer to do this. As for other architectures, you should be able to visualize the outputs of any layer you'd like, so it shouldn't be a problem.. – Vivek Reddy Dec 02 '20 at 21:57