I'm fairly new to Bayesian estimation with TensorFlow. I was trying to set up a very simple regression of height on weight (using McElreath's Howell data) to familiarize myself with the machinery in TensorFlow Probability, but I am running into something I don't understand. I presumed that defining a model with JointDistributionSequentialAutoBatched would yield a model that was identical to one defined by JointDistributionNamedAutoBatched, but the latter would just yield some nice handles for getting at parameters.
ht_wt: tfd.JointDistributionSequentialAutoBatched = tfd.JointDistributionSequentialAutoBatched([
tfd.Normal(loc=tf.cast(0., dtype=tf.float64), scale=0.2),
tfd.Normal(loc=tf.cast(0., dtype=tf.float64), scale=1.),
tfd.Uniform(low=tf.cast(1., dtype=tf.float64), scale=10.),
lambda beta0, beta1, sigma: tfd.Independent(tfd.Normal(
loc=beta0 + beta1 * weight, # weight is a tf.Tensor of weights from Howell
sigma=sigma
))
], name="Height vs Weight")
ht_wt_named: tfd.JointDistributionNamedAutoBatched = tfd.JointDistributionNamedAutoBatched([
beta0=tfd.Normal(loc=tf.cast(0., dtype=tf.float64), scale=0.2),
beta1=tfd.Normal(loc=tf.cast(0., dtype=tf.float64), scale=1.),
sigma=tfd.Uniform(low=tf.cast(1., dtype=tf.float64), scale=10.),
x=lambda beta0, beta1, sigma: tfd.Independent(tfd.Normal(
loc=beta0 + beta1 * weight, # weight is a tf.Tensor of weights from Howell
sigma=sigma
))
], name="Height vs Weight (Named)")
However, when I look at the distribution of logged probabilities across different parameter values, I get inconsistently different values. For example...
beta0 = 1., beta1 = 1., sigma = 0.5
yields:
Sequential = -19163.0633
Named = -inf
beta0 = 1., beta1 = 1., sigma = 1.0
yields:
Sequential = -5108.2755
Named = -5108.2755
beta0 = 1., beta1 = 1., sigma = 1.5
yields:
Sequential = -2616.9668
Named = -2601.3418
I get different values when I vary the beta as well. Have I missed something about underlying differences between Sequential and Named JointDistributions?
