How to distribute a file across presto cluster in kuebernetes

Question

I'm new to Kubernetes. We have a presto (starburst) cluster deployed in Kubernetes and we are trying to implement SSL certificate for the presto cluster.

Based on the below URL, I have created a keystore (in my local machine) and have to populate this keystore path to 'http-server.https.keystore.path'

https://docs.starburstdata.com/latest/security/internal-communication.html

However, this file has to be distributed across the cluster. If I enter the local path then Kubernetes is throwing 'file not found' error. Could you please let me know how to distribute this in presto cluster in kubernetes.

I have tried creating the keystore as secret and mounted this to a volume.

kubectl create secret generic presto-keystore --from-file=./keystore.jks

kind: Presto
metadata:
  name: stg-presto
spec:
  clusterDomain: cluster.local
  nameOverride: stg-presto
  additionalVolumes:
    - path: /jks
      volume:
      secret:
        secretName: presto-keystore
  additionalJvmConfigProperties: |
  image:
    name: xxxxx/presto
    pullPolicy: IfNotPresent
    tag: 323-e.8-k8s-0.20
  prometheus:
    enabled: true
    additionalRules:
      - pattern: 'presto.execution<name=TaskManager><>FailedTasks.TotalCount'
        name: 'failed_tasks'
        type: COUNTER
  service:
    type: NodePort
    name: stg-presto
  memory:
    nodeMemoryHeadroom: 30Gi
    xmxToTotalMemoryRatio: 0.9
    heapHeadroomPerNodeRatio: 0.3
    queryMaxMemory: 1Pi
    queryMaxTotalMemoryPerNodePoolFraction: 0.333
  coordinator:
    cpuLimit: "5"
    cpuRequest: "5"
    memoryAllocation: "30Gi"
    image:
      pullPolicy: IfNotPresent
    additionalProperties: |
      http-server.http.enabled=false
      node.internal-address-source=FQDN
      http-server.https.enabled=true
      http-server.https.port=8080
      http-server.https.keystore.path=/jks/keystore.jks
      http-server.https.keystore.key=xxxxxxx
      internal-communication.https.required=true
      internal-communication.https.keystore.path=/jks/keystore.jks
      internal-communication.https.keystore.key=xxxxxxx

Also tried creating config and mounted it as a volume. But still getting 'Caused by: java.io.FileNotFoundException: /jks/keystore.jks (No such file or directory)'.

Could you please let me know if am missing anything.

Thanks

As you are using Starburst's, the correct docs link is https://docs.starburstdata.com/latest/security/internal-communication.html (i've updated it above too, for posterity). — Piotr Findeisen, Jul 29 '20 at 21:02
Hi @PiotrFindeisen, I have tried the approach that Tarun has mentioned below 1. Created configMap kubectl create configmap presto-keystore --from-file=/cert/keystore.jks 2. Used below in the Presto yaml file additionalBootstrapScriptVolume: configMap: name: presto-keystore 3. Also, used properties for coordinator and workers in the same Presto yaml file. additionalProperties: | internal-communication.https.keystore.path=presto-keystore But still am getting 'file not found' issue. Could you please advise if am missing anything. Thanks — Kiran, Jul 30 '20 at 05:16

Tarun Khosla · Answer 1 · 2020-07-29T12:09:40.773

2

You can create a secret or Configmap using your keystore and mount it as volume and then use the path in your files.

How to create and use configMap in k8s here

How to configure a secret in k8s here

You can use both in a similar fashion in your Custom Resource as in any other resource. I see an option of additionalVolumes and documentation associated with it here

edited Jul 29 '20 at 12:09

answered Jul 29 '20 at 12:03

Tarun Khosla

1,274
7
10

score 2 · Answer 2 · answered Jul 30 '20 at 09:30

2

You can create a secret in K8s and mount it within Presto deployment using additionalVolumes property. Checkout documentation on additionalVolumes at https://docs.starburstdata.com/latest/kubernetes/presto_resource.html

answered Jul 30 '20 at 09:30

Karol Sobczak

21
1

score 0 · Answer 3 · answered Jul 30 '20 at 10:15

0

Create a secret from a file:

kubectl create secret generic cluster-keystore --from-file=./docker.cluster.jks

Add the secret in the "additionalVolumes" section in the yaml: (per Karol's URL above)

 additionalVolumes:
  - path: /jks
   volume:
    secret:
     secretName: "cluster-keystore"

Add the jks file to the coordinator "additionalProperties" section in your yaml:

  coordinator:
    cpuRequest: 25 
    cpuLimit: 25 
    memoryAllocation: 110Gi
    additionalProperties: |
      http-server.https.enabled=true
      http-server.https.port=8443
      http-server.https.keystore.path=/jks/docker.cluster.jks
      http-server.https.keystore.key=xxxxxxxxxxx
      http-server.authentication.type=PASSWORD

answered Jul 30 '20 at 10:15

Tom

1

Hi, I have followed the same steps mentioned by you. But still getting the same issue. Updated my question with the steps that I tried. Thanks – Kiran Jul 31 '20 at 05:57
Do the pods come up correctly? You can enter the coordinator pod by doing a: kubectl exec -it bash - then do a ls /jks and see if your jks file is there. I would also just start with the https server and not the internal communcation. In fact, that is pretty much overkill to have encryption on between pods. – Tom Aug 01 '20 at 10:43
Hi, I have checked the coordinator pod and I dont see /jks volume...not sure what am missing...Followed the same approach as mentioned above. Do we have to add the additional volume mentioned above into each container? Thanks – Kiran Aug 02 '20 at 06:40

How to distribute a file across presto cluster in kuebernetes

3 Answers3