The error message referring to service account, together with the fact that the project already exists, suggests that the googlecloudapiclient used by Ray Autoscaler is authenticated for a service account that doesn't have access to the project.
If this is true, then here's what I believe happens. Typically, when running Ray GCP Autoscaler, it will first check if the project with the given id exists. In your case, this request returns "not found" because there's no project with the given id associated with the service account. Now, because the project did not exist, Ray will automatically try to create one for you. Typically, if we created a new GCP project with a user account (i.e. non-service account), the newly created project would be associated with the user account's default organization. Service accounts, however, must specify a parent organization explicitly when creating a new project. If we look at the ray.autoscaler.config._create_project function, we see that the arguments passed to the projects.create method omit the 'parent' argument, which explains why you see the error.
To verify if this is true (and hopefully fix the problem), you could change the account used for authenticating with the googlecloudapiclient. I believe that the credentials used for the googlecloudapiclient requests are the same as used by the Google Cloud SDK, so you should be able to configure the accounts using the gcloud auth login command.
I think the Ray Autoscaler could be improved by either allowing user to explicitly specify the parent organization when creating a new project, or at least by providing a more elaborate error message for this particular case.
I hope this fixes your problem. If it doesn't, and you believe that that it's a problem with the Autoscaler, don't hesitate to open an issue or feature request to the Ray Issues page!
