3

I've been successfully using the Google cloud_sql_proxy on multiple Compute Engine instances for some time, until today, one instance at a time, the proxy started to show the following error pattern:

2017/05/30 13:28:07 New connection for "project-id-1234:us-central1:sql_instance"
2017/05/30 13:28:07 couldn't connect to "project-id-1234:us-central1:sql_instance": Post https://www.googleapis.com/sql/v1beta4/projects/project-id-1234/instances/sql_instance/createEphemeral?alt=json: stream error: stream ID 1; PROTOCOL_ERROR
2017/05/30 13:28:41 New connection for "project-id-1234:us-central1:sql_instance"
2017/05/30 13:28:41 Thottling refreshCfg(project-id-1234:us-central1:sql_instance): it was only called 33.490705951s ago
2017/05/30 13:28:41 couldn't connect to "project-id-1234:us-central1:sql_instance": Post https://www.googleapis.com/sql/v1beta4/projects/project-id-1234/instances/sql_instance/createEphemeral?alt=json: stream error: stream ID 1; PROTOCOL_ERROR

When trying to connect directly to MySQL (while using the proxy) I get error 2013 (HY000):

ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0 "Internal error/check (Not system error)"

What I've tried

  • Restarting the cloud_sql_proxy yielded a temporary fix until finally both my Compute Engine instances are unable to connect to my Cloud SQL instance and the proxies show only this result.
  • Restarting the Cloud SQL instance and both Compute Engine instances.
  • Eliminating the proxy: I added the appropriate networks to my SQL instance's Authorized Networks, and updated all applications to use the public IP. This restored functionality to my production apps, but now I'm using a public connection instead of local/proxy.

Some research

joe.holler
  • 180
  • 2
  • 7

1 Answers1

3

My team started seeing the same issue appear today, with GKE managed servers. Same as you saw: restarts of servers and DB did nothing.

We tried doing an update of the version of Google Cloud Proxy we were using from v1.05 to v1.09 and the problem went away (for now).

I know that's not much of an explanation but give it a try to see if that helps you.

Greg
  • 166
  • 1
  • 10
  • This was also our problem. Slightly frustrating that this just randomly starts happening. This brought down a production application of ours. – rstuart85 May 31 '17 at 04:45
  • After some debugging with Google, it seems whatever the issue was with v1.05 seems to be resolved now, however it is still recommended to use the latest version available. – joe.holler May 31 '17 at 14:21
  • had the same issue. After upgrade to 1.09 started working again. Thanks @Greg – Dmitri Safine May 31 '17 at 22:13