Kubernetes - how do I extract a secret tar file?

Question

I have mounted two tar files as secrets. I would like to mount them to my container and then unpack the contents. The commands that created the secrets are as follows:

kubectl create secret generic orderer-genesis-block --from-file=./channel-artifacts/genesis.block

kubectl create secret generic crypto-config --from-file=crypto-config.tar

kubectl create secret generic channel-artifacts --from-file=channel-artifacts.tar

The following is what I kubectl apply:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fabric-orderer-01
spec:
  selector:
    matchLabels:
      app: fabric-orderer-01
  replicas: 1
  template:
    metadata:
      labels:
        app: fabric-orderer-01
    spec:
      initContainers:
      - name: init-channel-artifacts
        image: busybox
        volumeMounts:
        - name: channel-artifacts
          mountPath: /hlf/channel-artifacts
        command: ['sh', '-c', 'tar -xf /hlf/channel-artifacts/channel-artifacts.tar']
      containers:
      - name: fabric-orderer-01
        image: hyperledger/fabric-orderer:1.4.9
        env:
        - name: ORDERER_CFG_PATH
          value: /hlf/
        - name: CONFIGTX_ORDERER_ADDRESSES
          value: "orderer.example.com:7050"
        - name: ORDERER_GENERAL_LISTENADDRESS
          value: 0.0.0.0
        - name: ORDERER_GENERAL_LISTENPORT
          value: "7050"
        - name: ORDERER_GENERAL_LOGLEVEL
          value: debug
        - name: ORDERER_GENERAL_LOCALMSPID
          value: OrdererMSP
        - name: ORDERER_GENERAL_GENESISMETHOD
          value: file
        - name: ORDERER_GENERAL_GENESISFILE
          value: /hlf/genesis.block
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: fabricfiles-01
          mountPath: /fabric
        - name: orderer-genesis-block
          mountPath: /hlf/
          readOnly: true
        - name: crypto-config
          mountPath: /hlf/crypto-config
          readOnly: true
        - name: channel-artifacts
          mountPath: /hlf/channel-artifacts
          readOnly: true
      volumes:
      - name: orderer-genesis-block
        secret:
          secretName: orderer-genesis-block
      - name: crypto-config
        secret:
          secretName: crypto-config
      - name: channel-artifacts
        secret:
          secretName: channel-artifacts
      - name: fabricfiles-01
        persistentVolumeClaim:
          claimName: fabric-pvc-01

My deployment succeeds, but when I bash into my pod, I don't see my tar files being extracted. I only see my tar files /hlf/channel-artifacts/channel-artifacts.tar and /hlf/crypto-config/crypto-config.tar. How should I go about extracting their contents?

moonkotte · Accepted Answer · 2021-05-11T15:53:20.550

When you create an initContainer and execute this command:

command: ['sh', '-c', 'tar -xvf /hlf/channel-artifacts/channel-artifacts.tar']

it runs in default for this container path. I checked this by adding pwd and ls -l commands.

Whole line is:

command: ['sh', '-c', 'tar -xvf /hlf/channel-artifacts/channel-artifacts.tar ; pwd ; ls -l']

From an initContainer you can get logs by:

kubectl logs fabric-orderer-01-xxxxxx -c init-channel-artifacts

Output was:

channel-artifacts.txt # first line for -v option so tar was untared indeed
/ # working directory
total 44
drwxr-xr-x    2 root     root         12288 May  3 21:57 bin
-rw-rw-r--    1 1001     1002            32 May 10 14:15 channel-artifacts.txt # file which was in tar
drwxr-xr-x    5 root     root           360 May 11 08:41 dev
drwxr-xr-x    1 root     root          4096 May 11 08:41 etc
drwxr-xr-x    4 root     root          4096 May 11 08:41 hlf
drwxr-xr-x    2 nobody   nobody        4096 May  3 21:57 home
dr-xr-xr-x  225 root     root             0 May 11 08:41 proc
drwx------    2 root     root          4096 May  3 21:57 root
dr-xr-xr-x   13 root     root             0 May 11 08:41 sys
drwxrwxrwt    2 root     root          4096 May  3 21:57 tmp
drwxr-xr-x    3 root     root          4096 May  3 21:57 usr
drwxr-xr-x    1 root     root          4096 May 11 08:41 var

As you can see your file is stored in / path of the container which means when this container is terminated, its filesystem is terminated as well and your file is gone.

Once we know what happened, it's time to workaround it. First and impotant thing is secrets are read-only and should be used in prepared form, you can't write file to secret like you wanted to do in your example.

Instead one of the options is you can untar your secrets to a persistent volume:

command: ['sh', '-c', 'tar -xvf /hlf/channel-artifacts/channel-artifacts.tar -C /hlf/fabric']

And then use postStart hook for the main container where you can e.g. copy your files to desired locations or create simlinks and you won't need to mount your secrets to the main container.

Simple example of postStart hook (reference):

apiVersion: v1
kind: Pod
metadata:
  name: lifecycle-demo
spec:
  containers:
  - name: lifecycle-demo-container
    image: nginx
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]

Small notice is

Kubernetes sends the postStart event immediately after the Container is created. There is no guarantee, however, that the postStart handler is called before the Container's entrypoint is called.

To workaround it you can add sleep 5 in your main container before entrypoint. Here's an example of a beginning of container section with nginx image (for your image it'll be different):

containers:
      - name: main-container
        image: nginx
        command: ["bash", "-c", 'sleep 5 ; echo "daemon off;" >> /etc/nginx/nginx.conf ; nginx']

This will fix your issue. Also you can use this approach for untar your files and you won't even need an initContainer

It's not clear why you want to use tar for this purpose as you can store small files in secrets or configmaps and mount them directly using subPath where they are needed without additional steps (you can read about it and find an example here)

To use secrets securely, you should consider e.g. HashiCorp Vault (Vault with kubernetes)

Kubernetes - how do I extract a secret tar file?

1 Answers1