0

I have a toy problem, where I have some data (X,Y) where the labels Y are frequencies, and X are cosine functions with frequency Y: X=cos(Y*t+phi)+N, where t is a time vector, phi is some random phase shift and N is additive noise. I am developing a CNN in keras (tensorflow backend) to learn Y from X. However, I don't know how long my time window needs to be. So, I would like to use keras-tuner to help identify the best hyperparameters (winStart,winSpan) to determine which times to select t[winStart:winSpan].

It is unclear if/how I can slice my learning features X using tuned hyperparameters.

First, I defined my data as:

# given sine waves X, estimate frequencies Y
t=np.linspace(0,1,1000)
Y=np.divide(2*np.pi,(np.random.random((100))+1)) 
X=np.transpose(np.cos(np.expand_dims(t, axis=1)*np.expand_dims(Y,axis=0)+np.ones((t.size,1))*np.random.normal(loc=0,scale=np.pi,size=(1,Y.size)))+np.random.normal(loc=0,scale=.1,size=(t.size,Y.size)))
Y=np.expand_dims(Y,axis=1)

Following tutorials, I have written a function to construct my CNN model:

def build_model(inputSize):   
    model = Sequential()
    
    model.add(Conv1D(10,
              kernel_size=(15,),
              padding='same',
              activation='ReLU',
              batch_input_shape=(None,inputSize,1)))
    
    model.add(MaxPool1D(pool_size=(2,)))
    model.add(Dropout(.2))
        
    model.add(Conv1D(10,
              kernel_size=(15,),
              padding='same',
              activation='ReLU',
              batch_input_shape=(None,model.layers[-1].output_shape[1],1)))
    
    model.add(MaxPool1D(pool_size=(2,)))
    model.add(Dropout(.2))    
    
    model.add(Flatten())
    
    # add a dense layer
    model.add(Dense(10))
    model.add(Dense(1))
           
    model.compile(loss='mean_squared_error',
                  optimizer='adam')
    
    return model

Additionally, I have written a hypermodel class:

class myHypermodel(HyperModel):
    def __init__(self,inputSize):
        self.inputSize=inputSize
        
    def build_hp_model(self,hp):
        inputSize=1000

        self.winStart=hp.Int('winStart',min_value=0,max_value=inputSize-100,step=100)
        self.winSpan=hp.Int('fMax',min_value=100,max_value=inputSize,step=100)
        
        return build_model(self.winSpan)    
    def run_trial(self, trial, x,y,*args, **kwargs):

        hp=trial.hyperparameters
        
        # build the model with the current hyperparameters
        model=self.build_hp_model(hp)

        # Window the feature vectors
        x=x[:,self.winStart:np.min([self.winStart+self.winSpan,self.inputSize])]

        print('here')
        return model.fit(x,y,*args,**kwargs)

Here, the build_hp_model() method is intended to link the hyperparameters to internal variables so that they can be used during when run_trial() method is called. My understanding is that run_trial() will be called by tuner.search() when performing hyperparameter optimization. I expect the run_trial() method to pick a new combination of winStart and winSpan hyperparameters, rebuild the model, remove all values of x except in the window defined by winStart and winSpan, and then run model.fit().

I call my hypermodel class and attempt to perform the hyperparameter search using:

tuner_model=myHypermodel(X.shape[1])

tuner = kt.Hyperband(tuner_model.build_hp_model,
                    overwrite=True,
                    objective='val_loss',
                    max_epochs=25,
                    factor=3,
                    hyperband_iterations=3)

tuner.search(x=np.expand_dims(X,axis=2),
             y=np.expand_dims(Y,axis=2),
             epochs=9,
             validation_split=0.25)

When I run the script, I get the error:

ValueError: Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 100, 1), found shape=(None, 1000, 1)

So it seems like the build_model() function is being called for a hyperparameter winSpan=100, but then the model is being fit using the full feature vectors X instead of X[:,winStart:winStart+winSpan].

Any suggestions on how I can properly implement this tuning?

john g.
  • 1

0 Answers0