First, define your summary over the required layer:
tf.summary.histogram("layer3_hist_summary", layer3)
Next define the summary writer which will be used to write your summaries to disk:
LOGDIR = 'path/to/logsdir' # define your required summary output folder
summary_file_writer = tf.summary.FileWriter(logdir=LOGDIR)
Assuming you may have multiple summaries, merge them to a single op:
summary_op = tf.summary.merge_all()
Now, in your training loop, write the summary result:
for i in range(NUM_ITR):
_, summary_res = sess.run([train_op, summary_op])
summary_file_writer.add_summary(summary_res, global_step=i)
To view these summaries, load tensorflow with logsdir=LOGDIR
.