How do you implement softmax with Tensorflow.JS

Question

Using Tensorflow.JS, I am trying to get a machine learning model running with a last dense layer using a softmax activation function. When I try to run it, I receive:

Error when checking target: expected dense_Dense2 to have shape [,1], but got array with shape [3,2].

If I comment out the fit function and run it, I get back a 1x2 array (as expected since I have 2 units on the last layer and I am only entering in one test.

Additionally, when I alter the y variables to this array: [[1,2,3]], it trains (but I don't think this is correct since the ys are not the correct shape of the last layer (2).

Any advice or help would be appreciated to fill in my gap of knowledge.

var tf = require('@tensorflow/tfjs');

let xs = tf.tensor([
  [1,2],
  [2,1],
  [0,0]
]);

let ys = tf.tensor([
  [1,2],
  [2,1],
  [1,1]
]);

async function createModel () {


  const model = tf.sequential();
  model.add(tf.layers.dense({inputShape: [2], units: 32, activation: "relu"}));
  model.add(tf.layers.dense({units: 2}));
  model.compile({loss: "sparseCategoricalCrossentropy",optimizer:"adam"});

  //await model.fit(xs, ys, {epochs:1000});

  model.predict(tf.tensor([[1,2]])).print();
}

createModel();

For binary classification (softmax of 2 values) should be better binaryCrossentropy and change the softmax to sigmoid. — Aral Roca, Mar 29 '21 at 17:07

score 4 · Accepted Answer · answered Mar 25 '20 at 18:49

Here is the softmax activation on the last layer:

  const model = tf.sequential();
  model.add(tf.layers.dense({inputShape: [2], units: 32, activation: "relu"}));
  model.add(tf.layers.dense({units: 2, activation:'softmax'}));
  model.compile({loss: "sparseCategoricalCrossentropy",optimizer:"adam"});

  model.predict(tf.tensor([[1,2]])).print();

For the error:

Error when checking target: expected dense_Dense2 to have shape [,1], but got array with shape [3,2].

You can consider the answer given here

Your error is related to the loss function sparseCategoricalCrossentropy which expect labels as tensor1d. If you change this loss function to categoricalCrossentropy, it will work. Both losses do the same thing, it is the way the labels are encoded that is different. But as it is in the question the labels are neither encoded for categoricalCrossentropy nor sparseCategoricalCrossentropy.

When using sparseCategoricalCrossentropy, the labels are a 1d tensor where the value are the index of the category
When using categoricalCrossentropy, the labels are a 2d tensor aka one-hot encoding

How do you implement softmax with Tensorflow.JS

1 Answers1