tensorflow - get max from tensor

Question

So I have a tensor h_in of shape (50, ?, 1, 100) that I should now like to turn into shape (50, 1, 1, 100) by taking the max over the axis 1.

How do I do that?

I tried

h_out = max_pool(h_in)

with

def max_pool(h,ksize=[1,-1,1,1],strides=[1,1,1,1],padding='VALID'):
    return tf.nn.max_pool(h,ksize=ksize,strides=strides,padding=padding)

but that doesn't seem to reduce the size.

runnable example:

import tensorflow as tf
import numpy as np
import numpy.random as nprand

def _weight_variable(shape,name):
    initial = tf.truncated_normal(shape,stddev=0.1)
    v = tf.Variable(initial,name=name)
    return v

def _bias_variable(shape,name):
    initial = tf.constant(0.1,shape=shape)
    v = tf.Variable(initial,name=name)
    return v

def _embedding_variable(shape,name):
    initial = tf.truncated_normal(shape)
    v = tf.Variable(initial,name=name)
    return v

def conv2d(x,W,strides=[1,1,1,1],padding='VALID'):
    return tf.nn.conv2d(x,W,strides=strides,padding=padding)

def max_pool(h,ksize=[1,-1,1,1],strides=[1,1,1,1],padding='VALID'):
    return tf.nn.max_pool(h,ksize=ksize,strides=strides,padding=padding)

nof_embeddings= 55000
dim_embeddings = 300

batch_size = 50
filter_size = 100
x_input = tf.placeholder(tf.int32, shape=[batch_size, None])

def _model():

    embeddings = _embedding_variable([nof_embeddings,dim_embeddings],'embeddings')

    h_lookup = tf.nn.embedding_lookup(embeddings,x_input)
    h_embed = tf.reshape(h_lookup,[batch_size,-1,dim_embeddings,1])

    f = 3

    W_conv1f = _weight_variable([f,dim_embeddings,1,filter_size],f'W_conv1_{f}')
    b_conv1f = _bias_variable([filter_size],f'b_conv1_{f}')
    h_conv1f = tf.nn.relu(conv2d(h_embed,W_conv1f) + b_conv1f)

    h_pool1f = max_pool(h_conv1f)

    print("h_embed:",h_embed.get_shape())
    print()
    print(f'h_conv1_{f}:',h_conv1f.get_shape())
    print(f'h_pool1_{f}:',h_pool1f.get_shape())
    print()

    return tf.shape(h_pool1f)

if __name__ == '__main__':

    tensor_length = 35

    model = _model()
    with tf.Session() as sess:
        tf.global_variables_initializer().run()
        batch = nprand.randint(0,nof_embeddings,size=[batch_size,tensor_length])
        shape = sess.run(model,
                         feed_dict ={
                                 x_input : batch
                                 })
        print('result:',shape)

which outputs

h_embed: (50, ?, 300, 1)

h_conv1_3: (50, ?, 1, 100)
h_pool1_3: (50, ?, 1, 100)

result: [ 50  35   1 100]

Let's say I instead hardcode the size that I want:

h_pool1f = max_pool(h_conv1f,ksize=[1,35-f+1,1,1])

That works. But now I'm in trouble as soon as I change the tensor_length (which is determined at runtime, so no, I cannot hardcode it).

One "solution" would be to blow the input up to a fixed maximum length by padding, or something, but then again, that introduces unnecessary computations and an artificial cap, both of which I should very much like to avoid.

So, is there

a way to make tensorflow "correctly" recognise the -1 in k_size?
or another way to compute the max?

@PietroTortella tested it for a bit, willing to believe it's doing what I'm looking for, thank you. Want to expand your comment into an answer? — User1291, Jun 08 '17 at 09:01

score 3 · Answer 1 · answered Jun 08 '17 at 11:15

I think tf.reduce_max is what you are looking for: https://www.tensorflow.org/api_docs/python/tf/reduce_max

Usage:

tens = some tensorflow.Tensor
ax = some positive integer, or -1 or None
red_m = tf.reduce_max(tens, axis=ax)

If tens has shape [shape_0, shape_1, shape_2], the resulting tensor red_m will have shape [shape_1, shape_2] if ax=0, shape [shape_0, shape_2] if ax=1, and so on. If ax=-1, the last axes is inferred, while if ax=None, the reduction will happen along all axes.

tensorflow - get max from tensor

1 Answers1