0

I am trying to implement Noisy Nets in my model. I found a code on GitHub which is an implementation of NoisyDense Class. I used this class inside my model. Here the code: -


class Agent:

    def __init__(self, state_size, strategy="t-dqn", reset_every=1000, pretrained=False, model_name=None):
        self.strategy = strategy

        # agent config
        self.state_size = state_size        
        self.action_size = 3                
        self.model_name = model_name
        self.inventory = []
        self.buffer = []
        self.first_iter = True
        self.nstep = 5
        self.n_step_buffer = deque(maxlen = self.nstep)
        self.cnt = count()
        self.alpha = 0.6
        self.model_name = model_name
        self.gamma = 0.95 # affinity for long term reward
        self.epsilon = 1.0
        self.epsilon_min = 0.01
        self.epsilon_decay = 0.995
        self.learning_rate = 0.001
        self.loss = huber_loss
        self.custom_objects = {"huber_loss": huber_loss}  # important for loading the model from memory
        self.optimizer = Adam(lr=self.learning_rate)

        if pretrained and self.model_name is not None:
            self.model = self.load()
        else:
            self.model = self._model()

        # strategy config
        if self.strategy in ["t-dqn", "double-dqn"]:
            self.n_iter = 1
            self.reset_every = reset_every

            # target network
            self.target_model = clone_model(self.model)
            self.target_model.set_weights(self.model.get_weights())
    def _model(self):
         model = Sequential()
         model.add(Dense(units=128, activation="relu", input_dim=self.state_size))
         model.add(Dense(units=256, activation="relu"))
         model.add(Dense(units=256, activation="relu"))
         model.add(NoisyDense(128, activation="relu"))
         model.add(NoisyDense(self.action_size + 1, activation='linear'))
         model.compile(loss=self.loss, optimizer=self.optimizer)
         return model

But ran into an error, I don't seem to know how to solve it, also Check NoisyDense implementation. I looked for similar issues on StackOverflow and I didn't get any solution that worked for my code. Also, the same model works when I don't call the NoisyDense class but use the Dense layer instead. Train.py file in the traceback calls the agent class nothing more important I guess. The model used is a Keras model and its version is 2.3.1

Traceback (most recent call last):
  File "train.py", line 85, in <module>
    pretrained=pretrained, debug=debug)
  File "train.py", line 52, in main
    agent = Agent(window_size, strategy=strategy, pretrained=pretrained, model_name=model_name)
  File "C:\Users\Desktop\agent.py", line 158, in __init__
    self.target_model = clone_model(self.model)
  File "D:\anaconda\lib\site-packages\keras\models.py", line 255, in clone_model
    return _clone_sequential_model(model, input_tensors=input_tensors)
  File "D:\anaconda\lib\site-packages\keras\models.py", line 207, in _clone_sequential_model
    layers = [clone(layer) for layer in model.layers]
  File "D:\anaconda\lib\site-packages\keras\models.py", line 207, in <listcomp>
    layers = [clone(layer) for layer in model.layers]
  File "D:\anaconda\lib\site-packages\keras\models.py", line 205, in clone
    return layer.__class__.from_config(layer.get_config())
  File "D:\anaconda\lib\site-packages\keras\engine\base_layer.py", line 1179, in from_config
    return cls(**config)
TypeError: __init__() missing 1 required positional argument: 'units'
Raj Shah
  • 13
  • 1
  • 4

2 Answers2

0

Class NoisyDense on Github as you can see takes unit as argument. As you are not giving this argument. That's why the error is showing

NaSir HuSSaiN
  • 192
  • 4
  • 12
  • is the first parameter not considered as units by defaults ```model.add(NoisyDense(128, activation="relu")) model.add(NoisyDense(self.action_size + 1, activation='linear'))``` – Raj Shah Mar 04 '20 at 06:34
  • well I also tried specifying it as units still the same error ``` model.add(NoisyDense(units = 128, activation="relu", sigma_init = 0.02))``` – Raj Shah Mar 04 '20 at 06:40
0

Thank you for your time but I got the answer from a similar question on StackOverflow

"ValueError: Unknown layer: ... " when calling copy.deepcopy(network) using Tensorflow

Raj Shah
  • 13
  • 1
  • 4