3

I'd like to create a [ ?, ?, n ] tensor at runtime. The data is as follow:

1) each element i'm examining is a vector of n elements (so [1,2,...,n])

2) each "group" is an unknown amount of elements of the previous type (so basically a matrix)

3) i don't know how many groups i will receive.

I tried manually, with something like this:

shape3 = [
[ [ .111,.112,.113 ], [ .121,.122,.123 ], [ .131,.132,.133 ] ],
[ [ .211,.212,.213 ], [ .221,.222,.223 ] ]
]

var_shape3 = tf.Variable(shape3, name="var_shape_3")
with tf.Session() as session:
    session.run(init_op)
    print var_shape3.eval()
    print var_shape3.get_shape()

but i receive the error

Argument must be a dense tensor: [[[0.111, 0.112, 0.113], [0.121, 0.122, 0.123], [0.131, 0.132, 0.133]], [[0.211, 0.212, 0.213], [0.221, 0.222, 0.223]]] - got shape [2], but wanted [2, 3, 3].

some help on what i'm doing wrong please?

on other words: how do i put those data in a tensor?

thank you

Stormsson
  • 1,391
  • 2
  • 16
  • 29

1 Answers1

3

In TensorFlow you can have dynamic dimensions, represented by the ? sign. But these dimensions must be inferred during execution, which means that it needs to be represented by a number once you execute your code.

In your example (with variable number of groups and elements in the groups), this will not work. E.g. what will work is:

shape3 = [
[ [ .111,.112,.113 ], [ .121,.122,.123 ], [ .131,.132,.133 ] ],
[ [ .211,.212,.213 ], [ .221,.222,.223 ], [ .000,.000,.000 ] ]
]

Your two options are:

  1. Define the maximum number of groups and elements in the groups and use padding to fill missing data points. You could also use bucketing to group similar sized examples (more info here).
  2. Change your code / data structure to work with variable length sequences. This would probably require using TensorFlow scan op. Note that this can be extremely slow, so I wouldn't recommend it, unless it's really required, more info here.
Community
  • 1
  • 1
Fematich
  • 1,588
  • 14
  • 26
  • Thank you for your answer, this leads me to ask how are variable input lengths handled? speaking generically in sentences for example you can't pad the text, or alter the input by putting it in buckets... – Stormsson Oct 21 '16 at 07:43
  • For sentences you will actually also use the same ideas (padding the most common one). By just adding "padding"-words and training your model with this kind of information, you should be able to get good performances. See also the TensorFlow [documentation on seq2seq models for sentence translations](https://www.tensorflow.org/versions/r0.11/tutorials/seq2seq/index.html#sequence-to-sequence-models), in which they also refer to padding and bucketing. – Fematich Oct 26 '16 at 22:45