limit the action value in Unity ML-Agents/Tensorflow

Question

I am using Unity with ML-Agents and their PPO implementation.

I have one Action to train my neural network on, which has an Imput of -1 to 1. When I log the action I can see that the Network always tries values like 550, 630,-530 etc. How can I limit these to only use values between -1 and 1?

I tried to look in Unity for it. Couldn't find any option. Now I am trying to modify the PPO algorithm, but I cannot find anything to limit my values.

My logging works like this: My Agent has the AgentStep method:

public override void AgentStep(float[] act){
  if (brain.brainParameters.actionSpaceType == StateType.continuous) {
    var actionAC = act[0];
    float[] toLog = new float[2];
    object.move(actionAC);
    // some rewards including toLog[0] as reward log
    toLog[1] = actionAC;
    logger.AddLine(toLog);
  }
}

Logger is a class written by me to just create a csv file. This output looks than like:

-1 530.73106
-2 530.73106
...
-234.5 -631.9137
...

thanks in advance.

Ah. I do have it in my unity outside of the Tensorflow code. I do have a class to write down a csv file with given data. In my Agent in the agentstep I then save my actionInput As a variable and call my csvwriter with that variable. — ChrizZlyBear, Feb 07 '18 at 13:39
@ChrizZlyBear it would be easier if you just showed us the code instead of describing it :) — Dunno, Feb 07 '18 at 14:01

score 1 · Answer 1 · answered Dec 16 '18 at 22:42

1

Try var actionAC = Mathf.Clamp(act[0], -1, 1);

This assures that the value of actionAC is always between -1 and 1.

https://docs.unity3d.com/ScriptReference/Mathf.Clamp.html

answered Dec 16 '18 at 22:42

Noodles

3,888
2
20
31

limit the action value in Unity ML-Agents/Tensorflow

1 Answers1