Handling numpy.exp overflow when using function on 2d-array

Question

I've got a 2d numpy array on which I want to use my function sigmoid(x) which is:

    def sigmoid(x):
        return 1 / (1 + np.exp(-x))

My problem is that I have inputs that are too big like 3000 and I get this warning:

RuntimeWarning: overflow encountered in exp
  return 1 / (1 + np.exp(-x/8.))

I tried to just assign values to inputs over a certain number like 700 -> 1 and -700 -> 0, however, this is very slow because I obv have to loop over the whole array that way.

I have also looked into np.logandexp(x1, x2) but I can't get it to work...

Edit: The datatype is float64 btw

Why not use [`scipy.special.expit`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.expit.html)? Alternatively, ignore or suppress the warnings; they're harmless, in that you should be getting the right floating-point result anyway. — Mark Dickinson, Apr 14 '19 at 13:17

score 0 · Answer 1 · answered Apr 14 '19 at 13:18

0

You can transform your input into the log space and run sigmoid after, this would shrunken down large values significantly.

answered Apr 14 '19 at 13:18

Ahmed Ragab

836
5
10

score 0 · Accepted Answer · answered Apr 14 '19 at 13:20

You can use SciPy's expit() function which is quite well behaved:

In [114]: from scipy.special import expit

# sample input array
In [115]: x = np.arange(50000, dtype=np.float64)

In [116]: sigm = expit(x)

# sanity check for no `np.inf`
In [117]: expit(70000.0)
Out[117]: 1.0

Handling numpy.exp overflow when using function on 2d-array

2 Answers2