Creating a simple ml-agent code (first attempt at this) with an agent who can move in the x or z direction not in the y along a flat plane. Code shows no errors and runs correctly initially (the only condition so far is that if it crosses a certain point in the z-direction it gets negative reward). For the first 20000 steps this works, before an error shows up:
"UserWarning: The use of x.T on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider x.mT to transpose batches of matrices or x.permute(*torch.arange(x.ndim - 1, -1, -1)) to reverse the dimensions of a tensor. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3281.)
return (tensor.T * masks).sum() / torch.clamp("
After this error, the ml agent environment continues functioning however performance becomes awful, as seen below. In all cases the agent flips direction eg) it was moving towards the bottom left corner of the plane and after this is moving to the top right corner at a much faster speed then previously. Image of this flip below ---
I have repeated this process multiple times and it is always on the 20000 step this error occurs. I have looked elsewhere for other answers of this question but found nothing.
This code may be useful:
