I developed a custom OpenAI-GYM environment and trained a CDQN model on it, now I am trying to figure out how can I test it not using my gym environment but in production (using real world observations), do you guys have any resources that can help me out?
Asked
Active
Viewed 170 times
3
-
Did you ever figure out best practice? I’m looking at similar issue. I’ve seen 2 use cases: 1.) take the model as a zip and just invoke model.predict with manually defining the observation data (in this case model inference is independent of model training) and 2.) have the gym environment interact with the real environment and deploy together with gym environment driving the model inference directly as in training. – jth_92 Jul 06 '22 at 15:44
-
Initially, it seemed like it would make more sense to have a separate deployment of the model independent of the environment since the real world interaction can vary with the state reference (eg different deployments with same environment context but the state data variables are accessed very differently). After doing that though, there is A LOT that can go wrong. – jth_92 Jul 06 '22 at 15:45
-
@jth_92 In the end I didn't have to deploy it in production, but anyway I think that the right orientation goes through the creation of the environment from the beginning. The environment and the state and action spaces should be exactly the input and output that you would have in the production environment. The key is that this forces any transformation you want to do to fall either on preprocessing objector between the environment and the agent or on the agent itself. In this way, the extraction of the agent and its implementation in production should be more straightforward. – Angelo Jul 11 '22 at 06:44