0

I'm using MLPs to forecast a time series, I implement a code that contain a mask layer to let the model skip the mask values.

for instance, in my data, the time series has a lot of NaN values, I fill it by a 'value = -999'. I don't want to remove it, but I want the Keras masking to skip it in gentle way.

My code as the following:

model = Sequential()
model.add(Masking(mask_value=-999, input_shape=(n_steps_in, )))
model.add(Dense(1024, activation='relu'))
model.add(Dense(n_steps_out))

I read an answer that said it is impossible to let masking works with MLPs.

How to add a mask layer for MLPs, or a custom mask layer to solve this problem?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Ahmad Aburoman
  • 21
  • 5
  • I've provided a detail answer below. But as an easy fix, you can set those values to 0 (unless 0 has other purpose in your problem)? – thushv89 Aug 11 '22 at 21:46
  • Thanks @thush89. I can't set them to zeros, zero meaning the value is equal to the sea level in my dataset, that's why I chose -999. I don't understand your answer very well, I'm new to LSTM and MLPs. my time series is timestamp column and sea level column; NaN values exist in sea level, and I fill them the value of -999 So I can use the mask. what you said is that I can't mask specific values in the sea level, is there another way to solve it? – Ahmad Aburoman Aug 12 '22 at 07:49
  • So why can't you use 0 for your `NaN` values (because it's a valid sea level i guess)? If so, just set missing values to 0 and transform your real sea values to be in a range (-inf, -1) and (1, inf), i.e. 0 doesn't appear in transformed data as a valid sea level. Can't you do that? – thushv89 Aug 13 '22 at 04:18

1 Answers1

0

What you're doing has a fundamental flaw.

The most important rule of masking is that,

the dimension(s) you're applying the mask on needs to stay without changing until the final prediction of the model

If that dimension(s) the mask is applied on changes, there's no way to propagate it forward.

Some scenarios masking will work,

  • If you want to mask specific batch items in an MLP. For example, in your MLP, you can mask a whole feature vector, but not a single feature in that vector. Here your mask would be a [batch size] size tensor and mask_value would be the same size as the feature vector.

  • If you want to mask a specific (batch item, time step) combination of an LSTM model, you can have the mask as [batch size, time step] sized tensor, where mask_value would have the same size as the feature vector of a time step.

So in summary, you can't mask only items, not specific features.

thushv89
  • 10,865
  • 1
  • 26
  • 39