3

I am trying to implement max drawdown for my loss function using code with the format of:

x = cumulative product of returns tensor
z = cumulative max of x
g = minimum of z / x

But I'm stuck on how to calculate cumulative maximum of x in Tensorflow. For example: given an array [0,2,5,3,8,1,7], the cumulative maximum of that array would be [0,2,5,5,8,8,8]. It creates an array with the max value so far.

Any tips would be greatly appreciated.

mikkola
  • 3,376
  • 1
  • 19
  • 41
Nubleet
  • 31
  • 2
  • 1
    I understand what a maximum of e.g. an array is. But what does the cumulative maximum mean? – mikkola Mar 16 '18 at 07:36
  • 3
    A cumulative maximum of an array would be for example: given an array [0,2,5,3,8,1,7] cummax of that array would output [0,2,5,5,8,8,8]. It creates an array with the max value so far. – Nubleet Mar 16 '18 at 09:10
  • It looks like it's solved here: https://stackoverflow.com/questions/34987509/tensorflow-max-of-a-tensor-along-an-axis – Veedoo Mar 16 '18 at 18:40
  • @Veedoo, it appears you accidentally posted a comment as an answer. You should delete the answer as it affects the likelihood that this question will be addressed by others. – David Parks Mar 16 '18 at 18:50
  • 1
    Note: question 34987509 referenced in comments above is for a different question, it isn't a solution to cumulative max. – David Parks Apr 29 '19 at 16:43

2 Answers2

5

Here's an implementation of cumulative_max using a tensorflow while loop which takes n=len(x) iterations. The code is copy-paste runnable on TF 2.x as an example.

import tensorflow as tf

def tf_while_condition(x, loop_counter):
  return tf.not_equal(loop_counter, 0)

def tf_while_body(x, loop_counter):
  loop_counter -= 1
  y = tf.concat(([x[0]], x[:-1]), axis=0)
  new_x = tf.maximum(x, y)
  return new_x, loop_counter

x = tf.constant([0,2,5,3,8,1,7])

cumulative_max, _ = tf.while_loop(cond=tf_while_condition, 
                                  body=tf_while_body, 
                                  loop_vars=(x, tf.shape(x)[0]))

print(cumulative_max)

Result:

[0 2 5 5 8 8 8]

Note: If you have a large vector to compute and you don't need backprop, it's probably worthwhile to include back_prop=False in the tf.while_loop.

A key to understanding TF while loops is to understand that your python based functions, tf_while_condition and tf_while_body, are only called once to produce the relevant tensorflow operations. Those two functions are NOT called in a loop. The operations they return will be executed in a loop within the tensorflow graph during sess.run computations.

David Parks
  • 30,789
  • 47
  • 185
  • 328
  • I ran your code in python and it works great, but I can't understand why the tensor 'y' isn't always [0,0,2,5,3,8,1]. I made tf_while_body return y to try and understand, but it was a tensor full of zeros... – Robert Bain Apr 29 '19 at 07:47
  • 1
    Notice that the tensor `y` is a shift operation. In iteration 1 `y=[0 0 2 5 3 8 1 7] x=[0 2 5 3 8 1 7]`, now the element-wise max of that is `z=[0 2 5 5 8 8 7]` and z is the new x in the next iteration. In iteration 2 `y=[0 0 2 5 5 8 8] x=[0 2 5 5 8 8 7]`, the element-wise max of those is now `z=[0 2 5 5 8 8 8]`, and z becomes the new x for the next iteration (notice the `return z, loop_counter` which is where `z` is feed back into the loop as `x`). After 2 iterations you can start to see the cumulative max evolving out of the shift/max opeations. Try it on paper, that's how I figured it out. – David Parks Apr 29 '19 at 16:20
  • 1
    Thank you so much for taking the time to explain that to me. I was missing the fact that z becomes the new x :) – Robert Bain Apr 29 '19 at 16:33
  • 1
    A small comment on David's excellent answer: if you are working inside `tf.function`, make sure to use `tf.shape(x)[0]` instead of `x.shape[0]` otherwise you will get a `None values not supported` error. – gdaras Oct 04 '21 at 16:50
  • 1
    @gdaras thanks for pointing that out. I updated the code with your suggested edit, and also updated it from TF 1.x to TF 2.x so it's runnable as-is on 2.x. – David Parks Oct 05 '21 at 22:12
2

The tf.scan op allows you to make all kinds of cumulative operations, a bit nicer solution might be the following (tested using TF 2.8.0)

tensor = tf.constant([0, 2, 5, 3, 8, 1, 7])
cumulative_max = tf.scan(lambda a, b: tf.maximum(a, b), tensor, initializer=tf.reduce_min(tensor))

# Result: [0, 2, 5, 5, 8, 8, 8]
pawlick2
  • 55
  • 7