0

I perform sigmoid which works fine but sigmoidDerivative gives the same result as sigmoid in nd4j. What is the difference between Transforms.sigmoidDerivative(x) and Transforms.sigmoidDerivative(x, true)?

INDArray x = Nd4j.create(new double[] { 0.1812, 0.1235, 0.8466 });
System.out.println(x);
System.out.println(Transforms.sigmoid(x));
System.out.println(Transforms.sigmoidDerivative(x));
System.out.println(Transforms.sigmoidDerivative(x, true));

Gives output: 

[[    0.1812,    0.1235,    0.8466]]
[[    0.5452,    0.5308,    0.6999]]
[[    0.5452,    0.5308,    0.6999]]
[[    0.2480,    0.2490,    0.2101]]

Comparing against python's numpy:

>>> def sigmoid(x):
...     return 1.0 / (1 + np.exp(-x))
... 
>>> def sigmoid_derivative(x):
...     a = sigmoid(x)
...     return a * (1.0 - a)

>>> x = np.array([    0.1812,    0.1235,    0.8466])
>>> sigmoid(x)
array([0.54517646, 0.53083582, 0.69985343])
>>> sigmoid_derivative(x)
array([0.24795909, 0.24904915, 0.21005861])

Nd4j pom:

<dependency>
     <groupId>org.nd4j</groupId>
     <artifactId>nd4j-native-platform</artifactId>
     <version>1.0.0-beta3</version>
</dependency>
Andronicus
  • 25,419
  • 17
  • 47
  • 88
absin
  • 1,116
  • 10
  • 21
  • FYI: Your implementation of `sigmoid_derivative(x)` is not correct. It should be `y = sigmoid(x); return y*(1 - y)` – Warren Weckesser Dec 27 '18 at 11:46
  • Something is wrong with your nd4j code, too. The third row of the output, which is supposed to be `sigmoidDerivative(x)`, is the same as the second row, which is `sigmoid(x)`. – Warren Weckesser Dec 27 '18 at 11:53
  • Facepalm! @WarrenWeckesser you are right my python implementation is wrong. – absin Dec 27 '18 at 11:58
  • Correcting my python code to : `>>> def sigmoid_derivative(x): ... a=sigmoid(x) ... return a * (1.0 - a) ` gives output: `array([0.24795909, 0.24904915, 0.21005861])` – absin Dec 27 '18 at 12:01
  • Actually changing `System.out.println(Transforms.sigmoidDerivative(x));` to `System.out.println(Transforms.sigmoidDerivative(x, true));` gives the expected sigmoid derivative `[[ 0.2480, 0.2490, 0.2101]]` Thanks solved! Editing the question – absin Dec 27 '18 at 12:01

1 Answers1

1

You are right, both Transforms.sigmoidDerivative(x) and Transforms.sigmoidDerivative(x, true) should give the same results, it's a bug in dl4j. The correct behavior has the latter method. I have submitted a pull request to resolve it.

Andronicus
  • 25,419
  • 17
  • 47
  • 88