Yes, it's easy to do in AWS. It can get a little complex as you sometimes have to write policies, but there are plenty of examples and tutorials around.
Look at the documentation for AWS Identity and Access Management (IAM). This allows you to give specific users access to specific resources at a very fine grained level. You use groups and policies to define who has access to what resources.
IAM Roles allow you when logged into your account to access resources in other accounts. Basically, an account sets up a trust relationship with your account, and you can assume a role to manage that account. It's slightly fiddly but it works fine.
I can't put all the information required in this answer, it's just too long. AWS documentation is excellent, plenty of tutorials.
As an aside, if you're managing multiple customers AWS accounts I suggest you use something like Linux Academy or Cloud Guru to study for at least the AWS Solution Architect Associate exam. All this is covered, and the professional goes even further. The qualification is good, but having the knowledge is key.
Updated Question re Billing
If you want the customer to pay the invoice directly you have to put each customer on their own account. This would typically mean creating an AMI, creating a new instance in the new account, then changing DNS and such.
If you bill the customer on behalf you could use cost allocation tags to work out the billing for each customer.
