differentiable maximum and minimum

Question

I need an approximation to maximum and minimum. Since the max and min are not differentiable I am looking for an differentiable approximation to it.

Does anybody know about it? for example I need to minimize the following:

[max (a - max_{x\in c}(x) )^2 + (a - max_{x\in d}(x) )^2]

max(a,b) = (a+b+abs(a-b))/2; min(a,b) = (a+b-abs(a-b))/2; so, just find a palatable approximation of *abs*. — Don Reba, Oct 28 '13 at 17:04
I have a vector of dimension d. in that case I need a differentiable approximation to max and min. — user570593, Oct 28 '13 at 17:13

score 2 · Answer 1 · answered Mar 08 '14 at 21:20

2

The Softmax function is a differentiable mapping from a vector to a scalar, and approximates the maximum function.

answered Mar 08 '14 at 21:20

Camille Goudeseune

2,934
2
35
56

score 1 · Answer 2 · answered Feb 27 '18 at 16:13

Smooth maximum (and minimum) is one candidate:

sum(x * exp(alpha * x)) / sum(exp(alpha * x))

where alpha -> +Inf converges to maximum, and alpha -> -Inf to minimum.

Another is LogSumExp:

log(sum(exp(x)))

Which is a max. I guess asking of a max of negated xs should give min

Couple of other are mentioned in paper Multiple Instance Learning: Algorithms and Applications such as generalized mean, noisy-or and the "ISR" model (because it's described in "Integrated segmentation and recognition" paper).

differentiable maximum and minimum

2 Answers2