Enhance a Machin Learning model for periodic data

Question

I am learning ML through TensorFlow (tfjs).
My first test was to train my model to predict cos(x) as a function of x (from 0 to 2*Math.PI*4 aka 4 periods)

feature: 2000 values of x (random)
label: 2000 values of cos(x)

model:

const model = tf.sequential({
    layers: [
        tf.layers.dense({ inputShape: [1], units: 22, activation: 'tanh' }),
        tf.layers.dense({ units: 1 }),
    ]
});

model.compile({
    optimizer: tf.train.adam(0.01),
    loss: 'meanSquaredError',
    metrics: ['mae']
});

...

await model.fit(feature, label, {
    epochs: 500,
    validationSplit: 0.2,
})

The result is quite "fun":

Now I would like to know how to enhance my model to fit with the periodicity nature of cos(x) (without using the mathematical periodicity of cos(x) like y = cos(x modulo 2PI) ).
Is it possible for my model to "understand" that there is a periodicity ?

score 1 · Answer 1 · answered Mar 10 '20 at 15:44

I think that the network you built is too small to learn the periodic behaviour of the cosine function (try increasing the number of hidden units and/or adding hidden layers), also I don't think a regular (fully connected neural network) is the right choice if you want to learn a function that has a periodic sequential nature, try using an RNN or LSTM for this.

Enhance a Machin Learning model for periodic data

1 Answers1