In one deep learning notes (Stanford cs20si), I once saw the following statement regarding eager. I don't quite understand what does the imperative custom layers indicate, and how to understand this code example in the context of imperative custom layers?
Normally, using tensorflow you are not able to access the content of a tensor directly. This means, that you are not able to use if-statements. Instead, you have to construct both possible branches of the branch and then use tf.conditional to include a node which switches between these two, depending on the content of a tensor. This makes it sometimes hard to implement imperative commands in layers.
The example you posted above show, that you are now (with eager execution) able to access the content of tensors, which means, that you can write all the if-statements, for-loops and so on, directly in python and you do not have to construct a huge graph on your own for each possibility. As the code inside the layer is now executed just like a normal imperative programming language, you can call this kind of layer an imperative layer - this is identical to the motivation behind PyTorch.
