Curriculum learning in ML Agents - YAML configuration

Question

I've set up a learning environment where a ball is to learn to go straight toward, and collide with, a target. The location of the target is randomised, but the radius of the target spawn point can be controlled by a public variable set in the gamecontroller in order to change the difficulty.

public float distanceToAgent = 1f;

My intention is to increase the distance through using lessons in the YAML curriculum - however when I try the below, it doesn't appear to increase the target spawn distance based on the lesson plan.

What am I doing wrong? I feel like I'm missing something in the script that links the lesson value to the curriculum configuration.

behaviors:
  RollerBall:
    trainer_type: ppo
    hyperparameters:
      batch_size: 10
      buffer_size: 100
      learning_rate: 3.0e-4
      beta: 5.0e-4
      epsilon: 0.2
      lambd: 0.99
      num_epoch: 3
      learning_rate_schedule: linear
    network_settings:
      normalize: false
      hidden_units: 128
      num_layers: 2
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
    max_steps: 900000
    time_horizon: 64
    summary_freq: 10000

environment_parameters:
  distanceToAgent:
    curriculum:
        - name: FirstLesson
          completion_criteria: 
            measure: progress
            behavior: RollerBall
            signal_smoothing: true
            min_lesson_length: 100
            threshold: 0.1
          value: 1.0

        - name: SecondLesson
          completion_criteria: 
           measure: progress
           behavior: RollerBall
           signal_smoothing: true
           min_lesson_length: 1000
           threshold: 0.3
          value: 5.0

        - name: ThirdLesson
          completion_criteria: 
           measure: progress
           behavior: RollerBall
           signal_smoothing: true
           min_lesson_length: 1000
           threshold: 0.5
          value: 10.0

        - name: ThirdLesson
          value: 15.0

score 0 · Answer 1 · answered Mar 24 '21 at 15:03

I think this is well.

But you need:

EnvironmentParameters m_ResetParams;
public override void Initialize()
    {
        m_ResetParams = Academy.Instance.EnvironmentParameters;
    }

//and than your can get you value for example in OnEpisodeBeginn()
distanceToAgent = m_ResetParams.GetWithDefault("distanceToAgent", 1.0f);

Curriculum learning in ML Agents - YAML configuration

1 Answers1