K8s Spring Application cannot connect to Mysql DB

Question

I wanted to try and deploy my spring boot application on kubernetes. I setted up a test environment with microk8s (dns,storage,ingress enabled) which consists of a pod running the application itself and a pod with the MySQL database. Each pod has its own service and is running on the same default namespace. The yaml files can be seen bellow:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: test-app
          image: myImage
          ports:
            - containerPort: 8080
          imagePullPolicy: Always
          env:
            - name: SPRING_APPLICATION_JSON
              valueFrom:
                configMapKeyRef:
                  name: spring-config
                  key: app-config.json

Application Service:

apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
  ports:
    - port: 8080
      targetPort: 8080

Mysql Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-server
  labels:
    # app: mysql
    app: mysql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      volumes:
        - name: mysql-persistent-volume-storage
          persistentVolumeClaim:
            claimName: mysql-pvc-claim
      containers:
        - name: mysql
          image: mysql
          volumeMounts:
            - name: mysql-persistent-volume-storage
              mountPath: /var/lib/mysql
              subPath: mysql-server
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: pass_root
            - name: MYSQL_USER
              value: user
            - name: MYSQL_PASSWORD
              value: pass
            - name: MYSQL_DATABASE
              value: test
          ports:
            - containerPort: 3306

Mysql Service:

apiVersion: v1
kind: Service
metadata:
  name: db-service
spec:
  selector:
    app: mysql
  ports:
    - port: 3306
      targetPort: 3306

For some reason my application can't use the database. It throws this error:

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
    at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) ~[mysql-connector-java-8.0.25.jar!/:8.0.25]
    at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) ~[mysql-connector-java-8.0.25.jar!/:8.0.25]
    at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:833) ~[mysql-connector-java-8.0.25.jar!/:8.0.25]
    at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:453) ~[mysql-connector-java-8.0.25.jar!/:8.0.25]
    at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246) ~[mysql-connector-java-8.0.25.jar!/:8.0.25]
    at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) ~[mysql-connector-java-8.0.25.jar!/:8.0.25]
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:121) ~[HikariCP-4.0.3.jar!/:na]
    at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364) ~[HikariCP-4.0.3.jar!/:na]
    at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) ~[HikariCP-4.0.3.jar!/:na]
    at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476) ~[HikariCP-4.0.3.jar!/:na]
    at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561) ~[HikariCP-4.0.3.jar!/:na]
    at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115) ~[HikariCP-4.0.3.jar!/:na]
    at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-4.0.3.jar!/:na]
.......

The application.yml:

db:
  datasource:
    url: jdbc:mysql://ip/test
    user: user
    password: pass
---
spring:
  datasource:
    url: ${db.datasource.url}
    username: ${db.datasource.user}
    password: ${db.datasource.password}
    driver-class-name: com.mysql.cj.jdbc.Driver
  jpa:
    database-platform: org.hibernate.dialect.MySQLDialect
  mvc:
    view:
      suffix: .html
  thymeleaf:
    cache: false
allowPublicKeyRetrieval: true
hibernate:
  show_sql: true
logging:
  level:
    org:
      hibernate:
        SQL: debug

I tried accessing the service from another pod on the namespace running mysql, since it has the mysql-client pre installed, and from the host. Both had access to database. I also tring ping on the pod running the application. It found the service withoyt any problem.

Then I tried using NodePort instead of ClusterIP. Nothing changed.

I made sure the credentials are correct.

Finally, I tried removing and adding the port in the application.yml.

I am completely stuck and I have no idea what's the problem. Any help would be appreciated.

Answer 1

I spot two problems in your configuration: The username/password in your mysql deployment does not match the values in your application.yaml.

The other one is that you use a property that spring boot does not use by default and I assume you have no special logic to handle that: "spring.data.url" should be "spring.datasource.url".

Note that it should be "spring.datasource.username" , not "user" as well.

Referring to your 'db' section in the YAML: In general I would not recommend to have a separate section for the database credentials and reference it using variables as well.