Deal with errors in parametrised sigmoid function in python

Question

I am trying to convert a set of numbers into sigmoids:

actualarray = {
    'open_cost_1':{
        'cost_matrix': [
            {'a': 24,'b': 56,'c': 78},
            {'a': 3,'b': 98,'c':1711},
            {'a': 121,'b': 12121,'c': 12989121},
        ]
    },
    'open_cost_2':{
        'cost_matrix': [
            {'a': 123,'b': 1312,'c': 1231},
            {'a': 1011,'b': 1911,'c':911},
            {'a': 1433,'b': 19829,'c': 1132},
        ]
    }
}

Where each number in each list of dicts in each cost_matrix gets normalised by different sigmoid functions:

def apply_normalizations(costs):

    def sigmoid(b,m,v):
        return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1 #Taken from http://web.stanford.edu/class/psych252/tutorials/Tutorial_LogisticRegression.html

    def normalize_dicts_local_sigmoid(bias, slope,lst):
        return [{key: sigmoid(bias, slope,val) for key,val in dic.iteritems()} for dic in lst]


    for name, value in costs.items():
        if int((name.split("_")[-1]))>1:
            value['normalised_matrix_sigmoid'] = normalize_dicts_local_sigmoid(0,1,value['cost_matrix'])


 apply_normalizations(actualarray)

However, when I run this, I get:

 RuntimeWarning: overflow encountered in exp
  return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1
 RuntimeWarning: invalid value encountered in double_scalars
  return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1

And the array becomes:

{
    'open_cost_2': {
        'cost_matrix': [
            {
                'a': 123,
                'c': 1231,
                'b': 1312
            },
            {
                'a': 1011,
                'c': 911,
                'b': 1911
            },
            {
                'a': 1433,
                'c': 1132,
                'b': 19829
            }
        ],
        'normalised_matrix_sigmoid': [
            {
                'a': 1.0,
                'c': nan,
                'b': nan
            },
            {
                'a': nan,
                'c': nan,
                'b': nan
            },
            {
                'a': nan,
                'c': nan,
                'b': nan
            }
        ]
    },
    'open_cost_1': {
        'cost_matrix': [
            {
                'a': 24,
                'c': 78,
                'b': 56
            },
            {
                'a': 3,
                'c': 1711,
                'b': 98
            },
            {
                'a': 121,
                'c': 12989121,
                'b': 12121
            }
        ]
    }
}

Note, every cost is always more than 0, hence I multiply by 2 and subtract 1 in my sigmoid function.

How can I adapt this to not have this error?

`return(RAM) (RAM)/ (RAM)` Solution: Change your computer or leave working with `TEMP` values ! — dsgdfg, Aug 29 '16 at 11:09
I don't quite get - is my function wrong? Can I replace nans with inf maybe? — Dhruv Ghulati, Aug 29 '16 at 12:29

score 1 · Accepted Answer · edited May 23 '17 at 10:33

1

As the warning states, the exponential in your implementation of the sigmoid function is overflowing. When that happens, the function returns nan:

In [3]: sigmoid(1000, 1, 1)
/Users/warren/miniconda3/bin/ipython:2: RuntimeWarning: overflow encountered in exp
  if __name__ == '__main__':
/Users/warren/miniconda3/bin/ipython:2: RuntimeWarning: invalid value encountered in double_scalars
  if __name__ == '__main__':
Out[3]: nan

Instead of writing your sigmoid function in terms of exp, you can use scipy.special.expit. It handles very large arguments correctly.

In [5]: from scipy.special import expit

In [6]: def mysigmoid(b, m, v):
   ...:     return expit(b + m*v)*2 - 1
   ...: 

In [7]: mysigmoid(1000, 1, 1)
Out[7]: 1.0

Check that it returns the same as your sigmoid function in cases where it doesn't overflow:

In [8]: sigmoid(1, 2, 3)
Out[8]: 0.99817789761119879

In [9]: mysigmoid(1, 2, 3)
Out[9]: 0.99817789761119879

See Numpy Pure Functions for performance, caching for my answer to another question about the sigmoid function.

edited May 23 '17 at 10:33

Community

1
1

answered Aug 29 '16 at 13:10

Warren Weckesser

110,654
19
194
214

Thanks @WarrenWeckesser. For the original function in my question, this is making every sigmoid value 0, even replacing the `'a': 1.0` value I had in my normalisation, the one that didn't normalise to `nan`. Logically all the very big numbers here should all make my sigmoid values 1 (given formula above) e.g. even `1231` should go to 1, not `nan` or 0. – Dhruv Ghulati Aug 29 '16 at 22:45
Sorry, I don't understand the problem. Could you give specific numerical values of `b`, `m` and `v` that don't give the value that you expect? For example, using your function, `sigmoid(0, 1, 1231)` generates a warning and returns `nan`. With my implementation, `mysigmoid(0, 1, 1231)` returns 1.0. – Warren Weckesser Aug 29 '16 at 23:33
If you run my code above with the example , you will see what happens - nothing goes to 1.0. – Dhruv Ghulati Aug 30 '16 at 09:01
It works for me. Did you *replace* your definition of `sigmoid()` with one that uses `expit`? – Warren Weckesser Aug 30 '16 at 13:24
Yes, `def sigmoid(b,m,v): return ((expit(b+m*v) / (1 + expit(b+m*v)))*2)-1` – Dhruv Ghulati Aug 30 '16 at 13:34
Please look at my definition of `mysigmoid()` again. Don't replace `exp` with `expit`. The return value is just `expit(b + m*v)*2 - 1`. – Warren Weckesser Aug 30 '16 at 13:37
Did you mean that is the whole definition, not `return ((expit(b+m*v) / (1 + expit(b+m*v)))*2)-1`? – Dhruv Ghulati Aug 30 '16 at 13:39
1

Yes. See the code marked `In [6]` in my ipython session. You would change the name to `sigmoid`, and use it instead of your current `sigmoid` function. – Warren Weckesser Aug 30 '16 at 13:40

Deal with errors in parametrised sigmoid function in python

1 Answers1