Appengine Managed VM deploy fails

Question

My managed VM was deployed and working fine. Then about a week ago it stopped working and started returning 500 for all requests. This is an Ubuntu container which is running a flask application.

The instances where showing as "restarting" in the developer console, but they were stuck that way for along time. When I tried deleting the instances I got an error messages saying the instances could not be deleted.

I tried to deploy the app again. The image creation process was a success, after which I got this error

    Updating service [abc]...failed.
    ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: The following quotas
    were exceeded: BACKEND_SERVICES (quota: 0, used: 0 + needed: 1),
    IN_USE_ADDRESSES (quota: 0, used: 0 + needed: 2), INSTANCES (quota:
    0, used: 0 + needed: 2).

The app has billing enabled, and a working credit card attached. There are not pending invoices. Does anyone know why this happens?

Have you tried deleting and recreating these instances using the gcloud command line ? — Sanket Berde, Mar 19 '17 at 09:33
@SanketBerde Yes I have, I was able to delete the existing instances finally and also deleted the current version, but when I try to deploy a new version I get this error — Procedurally Generated, Mar 19 '17 at 10:11
Make sure you have not set a too low billing limit in your project settings. — Sanket Berde, Mar 19 '17 at 10:26
@SanketBerde I increased the billing limit to $100 a day. The actual usage was less than $100 last month. Also in IAM & Admin Quotas, I can see some low usage for some google appengine, datastore and taskqueues. Most of the services have unlimited, and even the few with limits I'm about 1-2% of the limit. — Procedurally Generated, Mar 19 '17 at 11:05
@SanketBerde I saw a message yesterday saying my promotional period was expired and to hit the upgrade button to start billing for my app. The billing details were already updated, so I just hit the upgrade button and it said account upgraded. Maybe some issue with that? — Procedurally Generated, Mar 19 '17 at 11:07
@Varun Most likely. Also note that the effect of enabling billing or increasing the daily budget *might* not immediately clear overquota errors - for some billable items the periodic daily quota resets (or some other such job) needs to happen as well. Can't find that documentation note anymore, tho. — Dan Cornilescu, Mar 19 '17 at 14:03
@DanCornilescu I think that was it. It works now. Looks like I had to wait for some cron to reset that limit. I would think they could do that automatically when the account is upgraded. Thanks for all your help. — Procedurally Generated, Mar 20 '17 at 08:48

score 1 · Answer 1 · answered Mar 20 '17 at 08:53

I had my billing details updated in the account, but the account was using the promotional credits that I got as part of the signup. I saw a notification saying that promotional credits were expired and I needed to upgrade the account to continue using it. I hit the upgrade button and it said the account was upgraded.

It turned out that it takes a while for that to happen. Maybe a cron job had to run and reset some limits before I could deploy the app again.

It started working working fine after about a day.

Appengine Managed VM deploy fails

1 Answers1