3

I am building a model using Keras and Tensorflow probability that should output the parameters of a Gamma function (alpha and beta) instead of a single parameter as shown in the example below (t is passed to a Normal distribution function).

import tensorflow as tf
import tensorflow_probability as tfp
tfd = tfp.distributions

# Build model.
model = tf.keras.Sequential([
  tf.keras.layers.Dense(1),
  tfp.layers.DistributionLambda(lambda t: tfd.Normal(loc=t, scale=1)),
])

# Do inference.
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.05), loss=negloglik)
model.fit(x, y, epochs=500, verbose=False)

# Make predictions.
yhat = model(x_tst)

Instead of this I would like to output alpha and beta from two Dense layers and then pass this parameters to a Gamma distribution function.

Alberto
  • 467
  • 3
  • 16

2 Answers2

3

Something like this?

import tensorflow as tf
tf.enable_eager_execution()

print(tf.__version__) # 1.14.1-dev20190503

import tensorflow_probability as tfp
tfd = tfp.distributions

X = np.random.rand(4, 1).astype(np.float32)

d0 = tf.keras.layers.Dense(2)(X)
s0, s1 = tf.split(d0, 2)
dist = tfp.layers.DistributionLambda(lambda t: tfd.Gamma(t[0], t[1]))(s0, s1)

dist.sample() 
# <tf.Tensor: id=10580, shape=(2,), dtype=float32, numpy=array([1.1754944e-38, 1.3052921e-01], dtype=float32)>
muma
  • 345
  • 3
  • 10
  • I am not sure how exactly this would react if you were to train the model since tf.split will basically just make subset so for example a tensor of (1,5000) would create two (1,2500) tensor...I think having two independent dense layer would allow better learner of the alpha,beta params ! – Nico Coallier Jul 20 '20 at 22:26
  • 1
    Why? It shouldn't make a difference. Mathematically they are the same. The adjacency matrix from `d0` to `dist` will look the same. Using the `split` is also somewhat common, some VAE are trained this way as well - using the reparam-trick. – muma Jul 22 '20 at 06:48
-1
d0 = tf.keras.layers.Dense(2)(X)
dist = tfp.layers.DistributionLambda(lambda t: tfd.Gamma(t[:, 0], t[:, 1]))(d0)
James Risner
  • 5,451
  • 11
  • 25
  • 47
Andrew Polar
  • 1
  • 1