Tensor-flow how to use padding and masking layer in case of MLPs?

Question

I want to use MLPs to solve regression problem.

I have inputs with variable length to fix this I want to use Zero-padding with masking layer.

I read the inputs from csv file using pandas library. Here is how my data look like.

I know only how to fill NaN values with 0 using this command x_train.fillna(0.0).values

Like the first row :

[4, 0, 0, 512, 1.0, 0.0, 1.0, 0.0, 128.0 , NaN]

After padding :

[4, 0, 0, 512, 1.0, 0.0, 1.0, 0.0, 128.0 , 0.0]

The mask should be like this :

[1, 1, 1, 1, 1, 1, 1, 1, 1, 0]

But I do not know how to add the mask layer and feed them into my MLPs.

If i have fixed input length. My program will look like this

...
n_input = 10 #number og inputs

train_X = pd.read_csv('x_train.csv')
train_Y = pd.read_csv('y_train.csv')


X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_output])

...
y_pred = multilayer_perceptron(X)
...

with tf.Session() as sess:
    sess.run(init)

            _, c = sess.run([train, loss], feed_dict={X: train_X,
                                                      Y: train_Y})
          ...

I do not know how to combine between Zero padding and masking layer?

What do you want to do with the mask and the padded vector afterwards? — gorjan, Mar 20 '19 at 22:35
@gorjan, I have variable length inputs, so to fix this I'm using padding to fill up NaN values with Zero and I want to use masking to ignore the padded. — DINA TAKLIT, Mar 21 '19 at 14:27

score 2 · Accepted Answer · answered Mar 22 '19 at 13:23

2

You cannot ignore a single Features in an MLP. Mathematically we are talking about a matrix multiplication. The only dimensions you can "ignore" are time dimensions in recurrent layers since the number of weights does not scale with the dimension of time and so a single layer can take different sizes in the time dimension.

If you are only using Dense layers you cannot skip anything because your only dimension (besides the batch dimensions) scales directly with the number of weights.

answered Mar 22 '19 at 13:23

dennis-w

2,166
1
13
23

what you are saying right now is very important. Please please can we discuss on fb: https://web.facebook.com/zina.taklit.58 . I really want to omit all those ambiguities i have about padding and masking concept. – DINA TAKLIT Mar 22 '19 at 13:44
thank you. for you answer. I will do as we discuss. see the answer below. – DINA TAKLIT Mar 24 '19 at 16:13

score 0 · Answer 2 · answered Mar 24 '19 at 15:58

Thank you @dennis-ec your answer is very precise. I wanna just add this:

We can ignore all time steps for a given feature.This is is supported in Keras with LSTMs, but not Dense layers (We cannot ignore a single Features in an MLP)

we can suffice with padding (Zero padding or specify a value to use, e.g. -1) and see the performance.

Tensor-flow how to use padding and masking layer in case of MLPs?

2 Answers2

Linked