Can't load custom model in stablebasekines3

Question

I try to load a pretrained model in stablebaselines3, but I get this error instead.

C:\Users\prath\miniconda3\envs\rlunitybutler\lib\site-packages\stable_baselines3\common\save_util.py:166: UserWarning: Could not deserialize object policy_class. Consider using `custom_objects` argument to replace this object.
Exception: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from 'C:\\Users\\prath\\miniconda3\\envs\\rlunitybutler\\lib\\site-packages\\cloudpickle\\cloudpickle.py'>
  warnings.warn(
C:\Users\prath\miniconda3\envs\rlunitybutler\lib\site-packages\stable_baselines3\common\save_util.py:166: UserWarning: Could not deserialize object lr_schedule. Consider using `custom_objects` argument to replace this object.
Exception: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from 'C:\\Users\\prath\\miniconda3\\envs\\rlunitybutler\\lib\\site-packages\\cloudpickle\\cloudpickle.py'>
  warnings.warn(
C:\Users\prath\miniconda3\envs\rlunitybutler\lib\site-packages\stable_baselines3\common\save_util.py:166: UserWarning: Could not deserialize object clip_range. Consider using `custom_objects` argument to replace this object.
Exception: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from 'C:\\Users\\prath\\miniconda3\\envs\\rlunitybutler\\lib\\site-packages\\cloudpickle\\cloudpickle.py'>
  warnings.warn(
Wrapping the env in a DummyVecEnv.
Traceback (most recent call last):
  File "C:\Users\prath\Downloads\TeamProject_v6\python\trainagent.py", line 243, in <module>
    model = A2C.load("C:/Users/prath/Downloads/TeamProject_v6/python/modelv1/Model_v2", env=env, policy=CustomActorCriticPolicy, verbose=1,  n_steps=4096, batch_size=128, seed=51, tensorboard_log=f"C:/Users/prath/Downloads/TeamProject_v6/python/ppo", n_epochs=15)
  File "C:\Users\prath\miniconda3\envs\rlunitybutler\lib\site-packages\stable_baselines3\common\base_class.py", line 708, in load
    policy=data["policy_class"],
KeyError: 'policy_class'

Here is my code:

CustomNetwork is writing with the Pytorch-nn.model:

 class CustomActorCriticPolicy(ActorCriticPolicy):
    def __init__(
        self,
        observation_space: spaces.Space,
        action_space: spaces.Space,
        lr_schedule: Callable[[float], float],
        *args,
        **kwargs,
    ):

        super().__init__(
            observation_space,
            action_space,
            lr_schedule,
            # Pass remaining arguments to base class
            *args,
            **kwargs,
        )
        # Disable orthogonal initialization
        self.ortho_init = False

    def _build_mlp_extractor(self) -> None:
        self.mlp_extractor = CustomNetwork(self.features_dim)




if __name__ == '__main__':
    env = GoEnv()
    env = Monitor(env)

    model = PPO.load("C:/Users/prath/Downloads/TeamProject_v6/python/modelv1/Model_v2", env=env, policy=CustomActorCriticPolicy, verbose=1,  n_steps=4096, batch_size=128, seed=51, tensorboard_log=f"C:/Users/prath/Downloads/TeamProject_v6/python/ppo", n_epochs=15)

    for i in tqdm(range(4096*3*100)):
      # Perform a training step
      model.learn(total_timesteps=4096*3, progress_bar=True)
      model.save("C:/Users/prath/Downloads/TeamProject_v6/python/modelv1/agento]"+str(i))

I have tried to load from a .pkl file, but it didn't work.

model = PPO("C:/Users/prath/Downloads/TeamProject_v6/python/modelv1/agento_v1e5.pkl", env, verbose=1,  n_steps=4096, batch_size=64, seed=51, tensorboard_log=f"runs/ppo", n_epochs=15)
model.policy.load("C:/Users/prath/Downloads/TeamProject_v6/python/modelv1/agento.pkl")

There aren't any errors, but it seem like it does load the pretrained model.