How to build a Multiagent RL model with MLAgents toolkit in Unity3D?

Question

I am building a multiagent RL model with MLAgents toolkit in Unity3D.

The idea is, the model will have 2 agents(spheres) and 1 target(cube) on a platform(floor). Both agent need to reach the target before each other, if an agent will reach the goal before the other agent then it will receive a +ve reward and -ve reward to the other agent and if any of the agent will collide with the wall or reach out of the floor then it will receive -ve reward and 0 reward to the other agent.

I have done with the basic code. This code is able to move a single agent while training, I want to move both agents(multi agent) and compete with each other.

file name (Roller.cs)

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Unity.MLAgents;
using Unity.MLAgents.Actuators;
using Unity.MLAgents.Sensors;
using Palmmedia.ReportGenerator.Core.Reporting.Builders;
using System;
using Unity.Mathematics;
using System.Security.Cryptography;

public class Roller : Agent
{
    // Serialize field for target cube
    [SerializeField] Transform transformPrey;

    public Team teamId;
    public float[] preyZ = new float[5];

    public override void OnEpisodeBegin()
    {
        preyZ[0] = 0f;
        preyZ[1] = -2f;
        preyZ[2] = -4f;
        preyZ[3] = -3f;
        preyZ[4] = -6f;
        int RandomValue = UnityEngine.Random.Range(0, 5);
        transform.position = new Vector3(5.32f, 2.9f, -4.84f); 
        transformPrey.position = new Vector3(2.2f, 2.9f, preyZ[RandomValue]);

    }

    public override void CollectObservations(VectorSensor sensor)
    {   
        // trainsform is for player position
        sensor.AddObservation(transform.position);
        sensor.AddObservation(transformPrey.position); 

    }
    public override void OnActionReceived(ActionBuffers actions)
    {
        float moveX = actions.ContinuousActions[0];
        float moveZ = actions.ContinuousActions[1];

        float moveSpeed = 1f;
        transform.position += new Vector3(moveX, 0, moveZ) * Time.deltaTime * moveSpeed; 

    }
    private void OnTriggerEnter(Collider other)
    {   
        if (other.TryGetComponent<target>(out target trg))
        {
            SetReward(+1f);
            EndEpisode();
        }
        
        if (other.TryGetComponent<wall>(out wall we))
        {
            SetReward(-1f);
            EndEpisode(); 
        }
    }

    public override void Heuristic(in ActionBuffers actionsOut)
    {
        ActionSegment<float> continuousAction = actionsOut.ContinuousActions;
        continuousAction[0] = Input.GetAxisRaw("Horizontal");
        continuousAction[1] = Input.GetAxisRaw("Vertical"); 

    }
}

This is the picture of the scenario.

enter image description here

How to convert it into multiple agent system?

How to build a Multiagent RL model with MLAgents toolkit in Unity3D?

0 Answers0