After a lot of testing, I'm here with an answer on the Permission part when using PowerShell console to remove CM system object. Of course the symptom is the same: The account can delete from Admin console, however, when using PowerShell, it failed with Permission error message.
The account to perform the Remove-CMDevice cmdlet must have proper RBA Permission on the object. Assume the security scope is default one, the account connected to Configuration Manager console must have below RBA permission which I tested is almost minimal permission:

In the screenshot the Collection part, the permission is easy to understand, Read, Delete Resource, etc.
For the Computer Association part, you may get confused, why?
Steps I did the troubleshooting:
I opened a PowerShell Console connecting to Configuration Manager using my test account and run below command to see what will happen:
Remove-CMDevice 'Rsuraceccc' -Verbose
I got below error:

Yes, it's trying to querying from the SMS_StateMigration. Then I try to run a simple command:
Get-WMIObject -NameSpace root\sms\site_clt -Query 'Select * from SMS_StateMigration'
Once again, I got error. So I get the conclusion that the account needs Permission on SMS_StateMigration. So I add 'Recover User State' permission of Computer Association on the role and tried again, cheers, this time all command runs successfully.
I don't know why it's using SMS_StateMigration, but this is the case here.