I'm trying to create a composite covariance function to model my data. Specifically I want to create a kernel which is weighted between @covSEard & @covRQard. For ex: I want to give a 30% weight to @covSEard & 70% to @covRQard, something like 0.3*@covSEard + 0.7*@covRQard
I tried the following options.
Option 1 - Using the @covSum & @covProd functionalities
input_dimensions = 4; % number of input dimensions
seed = 1234; % seed for reproducibility
rng(seed);
X = rand(100, input_dimensions); % sample input data generation
y = rand(100, 1); % sample output data generation
addpath(genpath(PATH_TO_GPML_TOOLBOX)); % adding gpml toolbox to the path
meanFunc = {@meanZero}; % using a zero mean
meanHyp = {[]}; % zero mean has no hyper parameters
kernel_weight = 0.3; % weight of covSEard kernel, 0.7 will be the weight for covRQard kernel
% defining the covariance function as a weighted sum of covSEard & covRQard
covFunc = {'covSum', {{'covProd', {'covConst','covSEard'}}, {'covProd', {'covConst','covRQard'}}}};
% variables to define the hyperparameters for the above kernel
sf=2; L=rand(input_dimensions,1); al=2;
% Covariance function for a constant function. The covariance function is parameterized as:
% k(x,z) = sf^2
%
% The scalar hyperparameter is:
% hyp = [ log(sf) ]
covHyp = {log([sqrt(kernel_weight); L; sf; sqrt(1-kernel_weight); L; sf; al])};
likFunc = {@likGauss}; % Using a gaussian likelihood
likHyp = {-1}; % Likelihood hyper parameter initialization
infMethod= @infGaussLik; % Using Gaussian inference
iters = -300; % Number of iterations for Bayesian Optimization
% Defining the hyper parameter struct
hyp.lik = cell2mat(likHyp(1));
hyp.cov = cell2mat(covHyp(1));
hyp.mean = cell2mat(meanHyp(1));
% Defining mean, covariance and likelihood functions
mF = meanFunc{1,1}; cF=covFunc; lF=likFunc{1,1};
hyp2vfe = minimize(hyp, @gp, iters, infMethod, mF, cF, lF, X, y); % Optimization of hyperparameters / Training
[nll, ~] = gp(hyp2vfe, infMethod, mF, cF, lF, X, y); % Negative Log Likelihood calculation
Here, I'm trying to use the @covConst kernel which has only the signal variance hyper-parameter which I'm forcing to be equal to the weight (Ex: 0.3 for @covSEard & 0.7 for @covRQard in this case.
However, when I try to optimize the above kernel's hyper parameters, even the weights (which are actually the hyper parameters of the @covConst kernel are also getting modified.
Option 2 - Using the @covSum functionality and repeating each kernel n times based on weight
For example, if I want to give 1:2 weights to @covSEard & @covRQard respectively, I would do the following
replace the following lines in the above code
covFunc = {'covSum', {{'covProd', {'covConst','covSEard'}}, {'covProd', {'covConst','covRQard'}}}};
with
covfunc = {'covSum', {'covSEard', 'covRQard', 'covRQard'}};
&
covHyp = {log([sqrt(kernel_weight); L; sf; sqrt(1-kernel_weight); L; sf; al])};
with
covHyp = {log([L; sf; L; sf; al; L; sf; al])};
However, in this case the number of hyper parameters would increase linearly and also I'm not sure if this is the right way to do things.
I want to know what is the correct way to create such a covariance function in GPML. Please suggest.
