GPflow, bvh: ValueError: mean must be 1 dimensional

Question

I am having a weird "ValueError: mean must be 1 dimensional" when I am trying to build a Hierarchical GL-LVM model. Basically I'm trying to reproduce this paper: Hierarchical Gaussian Process Latent Variable Models using GPflow.

Therefore I implemented my own new model as follow:

class myGPLVM(gpflow.models.BayesianModel):
    def __init__(self, data, latent_data, x_data_mean, kernel):
        super().__init__()
        print("GPLVM")
        self.kernel0 = kernel[0]
        self.kernel1 = kernel[1]
        self.mean_function = Zero()
        self.likelihood0 = gpflow.likelihoods.Gaussian(1.0)
        self.likelihood1 = gpflow.likelihoods.Gaussian(1.0)
        # make some parameters
        self.data = (gpflow.Parameter(x_data_mean), gpflow.Parameter(latent_data), data)

    def hierarchy_ll(self):
        x, h, y = self.data
        K = self.kernel0(x)
        num_data = x.shape[0]
        k_diag = tf.linalg.diag_part(K)
        s_diag = tf.fill([num_data], self.likelihood0.variance)
        ks = tf.linalg.set_diag(K, k_diag + s_diag)
        L = tf.linalg.cholesky(ks)
        m = self.mean_function(x)

        return multivariate_normal(h, m, L)

    def log_likelihood(self):
        """
        Computes the log likelihood.

        .. math::
            \log p(Y | \theta).

        """
        x, h, y = self.data
        K = self.kernel1(h)
        num_data = h.shape[0]
        k_diag = tf.linalg.diag_part(K)
        s_diag = tf.fill([num_data], self.likelihood1.variance)
        ks = tf.linalg.set_diag(K, k_diag + s_diag)
        L = tf.linalg.cholesky(ks)
        m = self.mean_function(h)

        # [R,] log-likelihoods for each independent dimension of Y
        log_prob = multivariate_normal(y, m, L).  # <- trows the error!
        log_prob_h = self.hierarchy_ll()
        log_likelihood = tf.reduce_sum(log_prob) + tf.reduce_sum(log_prob_h)
        return log_likelihood

The model seems to work with a toy example:

    from sklearn.datasets.samples_generator import make_blobs
    X, y = make_blobs(n_samples=40, centers=3, n_features=12, random_state=2)
    Y = tf.convert_to_tensor(X, dtype=default_float())

but fails and trough me the error when I am trying with a bvh file (the one from the paper actually). I also used Lawrence's code to read my bvh from mocap which I modified to fit python3

Anyway, it's been few a days and I am out of ideas. I tried multiple way to force my mean array "m" to be of one dimensional but nothing worked. I also tried with the "three_phase_oil_flow" dataset from the first GPLVM paper which works as well.

Therefore, I would assume that my model is correct, or at least I got some optimisation going on, and would think that perhaps the bvh reader could be the cause. But the data seems all fine to me... Especially I don't understand why when forcing multivariate function like:

m = np.zeros((np.shape(m)[0], 1))
log_prob = multivariate_normal(y, m, L)

or even with the gpflow Zero function

m = Zero(h)
log_prob = multivariate_normal(y, m, L)

it still trows me the error. Any help will be highly appreciated.

edited thanks to: Artem Artemev The rest of the code if anyone wants to try to reproduce: https://github.com/michaelStettler/h-GPLVM

error flow:

(venv) MacBookMichael2:stackOverflow michaelstettler$ python3 HGPLVM.py 
(199, 96)
shape Y (199, 3, 38)
2020-01-26 17:00:48.104029: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-01-26 17:00:48.113609: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8dd5ff5410 executing computations on platform Host. Devices:
2020-01-26 17:00:48.113627: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
shape Y (199, 38)
Number of points: 199 and Number of dimensions: 38
shape x_mean_latent (199, 8)
shape x_mean_init (199, 2)
HGPLVM
gpr_data (199, 2) (199, 8) (199, 38)
2020-01-26 17:00:48.139003: W tensorflow/python/util/util.cc:299] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
shape m (199, 1)
Traceback (most recent call last):
  File "HGPLVM.py", line 131, in <module>
    _ = opt.minimize(closure, method="bfgs", variables=model.trainable_variables, options=dict(maxiter=maxiter))
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/gpflow/optimizers/scipy.py", line 60, in minimize
    **scipy_kwargs)
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/scipy/optimize/_minimize.py", line 594, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 998, in _minimize_bfgs
    gfk = myfprime(x0)
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 327, in function_wrapper
    return function(*(wrapper_args + args))
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 73, in derivative
    self(x, *args)
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 65, in __call__
    fg = self.fun(x, *args)
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/gpflow/optimizers/scipy.py", line 72, in _eval
    loss, grads = _compute_loss_and_gradients(closure, variables)
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/gpflow/optimizers/scipy.py", line 116, in _compute_loss_and_gradients
    loss = loss_cb()
  File "HGPLVM.py", line 127, in closure
    return - model.log_marginal_likelihood()
  File "/Users/michaelstettler/PycharmProjects/GPflow/venv/lib/python3.6/site-packages/gpflow/models/model.py", line 45, in log_marginal_likelihood
    return self.log_likelihood(*args, **kwargs) + self.log_prior()
  File "HGPLVM.py", line 62, in log_likelihood
    log_prob = multivariate_normal(y, m, L)
  File "mtrand.pyx", line 3729, in numpy.random.mtrand.RandomState.multivariate_normal
ValueError: mean must be 1 dimensional

If your code works fine with other datasets, just not the one coming out from your bvh reader, then it would probably be helpful to investigate the shapes of the numpy arrays returned by that reader - this does not seem like a GPflow issue (all GPflow requires is rank-2 numpy arrays, X should have shape `(num_data, input_dim)`). — STJ, Jan 27 '20 at 12:09
Yes I did, and the input where completely the same, the issue is that once with my toy example my code is calling as expected the multivariate_normal from gpflow, but once I switched to the bvh reader, it called the multivariate_normal from that "mtrand.pyx". Which I still don't know why though. But anyway, thanks to Artem and the way he called the function in his example, it struck me and I am now forcing my code to call the multivariate_normal from gpflow all the time — Michael Stettler, Jan 27 '20 at 15:35

score 1 · Accepted Answer · answered Jan 20 '20 at 14:47

1

I would recommend posting a working MWE code. I have tried to use your code snippets, but it gives me errors.

I don't have issues with multivariate_normal function. If you have localised the issue correctly you can debug TF2.0 more thoroughly and find the place that causes that exception. Here is the code which I'm running:

In [2]: from sklearn.datasets.samples_generator import make_blobs
   ...: X, y = make_blobs(n_samples=40, centers=3, n_features=12, random_state=2)
In [10]: m = np.zeros((np.shape(y)[0], 1))
In [11]: m.shape
Out[11]: (40, 1)
In [12]: y.shape
Out[12]: (40,)
In [13]: L = np.eye(m.shape[0])
In [15]: gpflow.logdensities.multivariate_normal(y, m, L)
Out[15]:
<tf.Tensor: shape=(40,), dtype=float64, numpy=
array([ -56.75754133,  ...])>

answered Jan 20 '20 at 14:47

Artem Artemev

516
3
8

Thanks @Artem Artemev for taking time on my issue. Indeed with the blob function I also make it works, but not using the the bvh reader. I just made a small repo where you will be able to try my code using the data: https://github.com/michaelStettler/h-GPLVM let me know if you can reproduce the error – Michael Stettler Jan 22 '20 at 09:56
1

I'm sorry, @MichaelStettler, but I don't have time to review the whole repository. As I advised before, it would be better if you made a gist or a short code snippet with the code which **runs** and doesn't work as you expect. Thanks! – Artem Artemev Jan 24 '20 at 11:11
Well that is exactly the whole difficulty for me here, the code run with my toy example using the make_blobs function, but as soon as I change to the bvh reader it throws me that error. That is why I have created that minimalist repository where the code run and call the reader. Therefore I cannot post the code as a gist or a small working snipet. I will update my code with the entire error flow, perhaps there's something in there that I have missed. – Michael Stettler Jan 26 '20 at 15:57
Well then actually thanks a lot because by seeing again your code and copy pasting your line: gpflow.logdensities.multivariate_normal I realised that I had to force calling the gpflow library function, which made it worked. But I don't really get why it worked with the toy example and not with the bvh reader.. – Michael Stettler Jan 26 '20 at 16:30

GPflow, bvh: ValueError: mean must be 1 dimensional

1 Answers1