we are trying to create a data proc cluster with a newer image. We already upgraded our cloud SQL backed hive metastore to 3.1.0 . But while creating dataproc cluster with dataproc initialization action (where we are installing cloud sql proxy ) its failing with an error saying "Failed to bring up Cloud SQL Metastore"
below is the snippet of log trace
Synchronizing state of mysql.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable mysql
Removed /etc/systemd/system/multi-user.target.wants/mysql.service.
Created symlink /etc/systemd/system/multi-user.target.wants/cloud-sql-proxy.service → /lib/systemd/system/cloud-sql-proxy.service.
About to run 'nc -zv localhost 3306' with retries...
nc: connect to localhost port 3306 (tcp) failed: Connection refused
Connection to localhost 3306 port [tcp/mysql] succeeded!
Cloud SQL Proxy installation succeeded
About to run 'run_validation' with retries...
Metastore connection URL: jdbc:mysql://localhost:3306/hive_metastore
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hive
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
Hive distribution version: 3.1.0
Metastore schema version: 3.1.0
schemaTool completed
[2022-09-29T12:46:49+0000]: Failed to bring up Cloud SQL Metastore
As you can see schema is matching but still I am getting errors.
as part of the backup, I have cloned the previous cloud SQL instance and upgraded that cloud SQL instance to 3.1.0 and using that clone instance as metastore in the new dataproc cluster.
can anybody help me to understand this problem and solution to this problem?
