I am putting together a proof-of-concept using the latest spring boot, HashiCorp Consul & Connect. Basically, I want to show how webflux web services can use Consul for configuration and use Consul's mesh for secured communications. So, the latter is working with the following configuration, but the webflux service cannot get its configuration from Consul. I assume that the settings in bootstrap.yml for Spring Cloud Consul Config would be to the sidecar, but the connection fails. I've tried a lot of alternatives, and have hit a dead end. I'd appreciate any suggestions on how to resolve this issue.
Thank you for your help and time! Mike
=================== ORIGINAL INFORMATION =====================
# Dependencies
extra["springCloudVersion"] = "Hoxton.SR9"
dependencies {
implementation("org.springframework.boot:spring-boot-starter-actuator")
implementation("org.springframework.boot:spring-boot-starter-webflux")
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")
implementation("org.springframework.cloud:spring-cloud-starter-consul-all")
}
# Error when the service launches in k8s
Setting Active Processor Count to 8
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx1661599K -XX:MaxMetaspaceSize=128352K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 2G, Thread Count: 50, Loaded Class Count: 20247, Headroom: 0%)
Adding 138 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -agentpath:/layers/paketo-buildpacks_bellsoft-liberica/jvmkill/jvmkill-1.16.0-RELEASE.so=printHeapHistogram=1 -XX:ActiveProcessorCount=8 -XX:MaxDirectMemorySize=10M -Xmx1661599K -XX:MaxMetaspaceSize=128352K -XX:ReservedCodeCacheSize=240M -Xss1M -Dorg.springframework.cloud.bindings.boot.enable=true
2020-11-12 01:15:15.066 INFO 1 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.retry.annotation.RetryConfiguration' of type [org.springframework.retry.annotation.RetryConfiguration$$EnhancerBySpringCGLIB$$ef83abed] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.5.RELEASE)
2020-11-12 01:15:22.779 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 01:15:23.783 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 01:15:24.885 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 01:15:26.097 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 01:15:27.432 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 01:15:28.900 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 01:15:28.917 ERROR 1 --- [ main] o.s.boot.SpringApplication : Application run failed
com.ecwid.consul.transport.TransportException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8501 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
# Relevant bootstrap.yml
server:
port: ${PORT:8080}
spring:
application:
name: k8sconsulwebflux
profiles: development
cloud:
consul:
# Using Consul Connect Service Mesh
scheme: https
host: localhost
port: 8501
# Custom Helm Setting for Hashi Consul
global:
name: "consul-"
image: "consul:1.8.5"
imageK8S: "hashicorp/consul-k8s:latest"
domain: consul
datacenter: csc
gossipEncryption:
secretName: "gossip-encryption-key"
secretKey: "key"
tls:
enabled: true
enableAutoEncrypt: true
acls:
manageSystemACLs: true
lifecycleSidecarContainer:
resources:
requests:
memory: "25Mi"
cpu: "20m"
limits:
memory: "50Mi"
cpu: "20m"
imageEnvoy: "envoyproxy/envoy-alpine:v1.14.4"
server:
replicas: 1
bootstrapExpect: 1
disruptionBudget:
enabled: true
maxUnavailable: 0
resources:
requests:
memory: "100Mi"
cpu: "100m"
limits:
memory: "100Mi"
cpu: "100m"
client:
enabled: true
grpc: true
resources:
requests:
memory: "100Mi"
cpu: "100m"
limits:
memory: "100Mi"
cpu: "100m"
updateStrategy: |
rollingUpdate:
maxUnavailable: 2
type: RollingUpdate
# Enable the Consul Web UI via a NodePort
ui:
enabled: true
service:
type: 'NodePort'
syncCatalog:
enabled: true
default: true
consulPrefix: "k8s-"
k8sPrefix: "consul-"
resources:
requests:
memory: "50Mi"
cpu: "50m"
limits:
memory: "50Mi"
cpu: "50m"
logLevel: debug
# Enable Connect for secure communication between nodes
connectInject:
enabled: true
default: true
resources:
requests:
memory: "50Mi"
cpu: "50m"
limits:
memory: "50Mi"
cpu: "50m"
centralConfig:
enabled: true
sidecarProxy:
resources:
requests:
memory: 100Mi
cpu: 100m
limits:
memory: 100Mi
cpu: 100m
initContainer:
resources:
requests:
memory: "25Mi"
cpu: "50m"
limits:
memory: "150Mi"
cpu: "50m"
=================== UPDATED INFORMATION 1 =====================
After making the changes that Blake suggested, my app has moved passed the initial issue and on to a weird TLS/SSL issue... Arggg! I'd appreciate any suggestions. At this point, Consul is configured to handle the ssl configuration, so I am surprised about the error... This error is happening in my minikube "cluster" and in a real k8s cluster at AKS.
I've published the simple, proof-of-concept demo at gitlab, in case someone would like to take a closer look:
https://gitlab.com/abitofhelp.shared/spring-consul-proxy-k8s.git
:: Spring Boot :: (v2.3.5.RELEASE)
2020-11-12 06:55:42.529 ERROR 1 --- [ main] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2020-11-12 06:55:42.541 ERROR 1 --- [ main] o.s.boot.SpringApplication : Application run failed
com.ecwid.consul.transport.TransportException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at com.ecwid.consul.transport.AbstractHttpTransport.executeRequest(AbstractHttpTransport.java:83) ~[consul-api-1.4.5.jar:na]
