0

I'm running an operator in a Kind cluster. My development machine runs Docker for Mac for that Kind cluster to run. One of my operator function is about executing a command in a running Pod - similar to kubectl exec.

When the operator is running without Kind i.e. k3s on Linux host, I can stream the Pod's logs. Something like this:

--- start of exec pod logs ---
/home/ubuntu
--- end of exec pod logs ---

But when it's running in Kind cluster, I no longer can see the Pod's logs. All I see is just:

--- start of exec pod logs ---
--- end of exec pod logs ---

Here is my operator code (Go):

import (
    "sigs.k8s.io/controller-runtime/pkg/client/config"
    "k8s.io/client-go/tools/remotecommand"
    "k8s.io/client-go/kubernetes"
    "k8s.io/api/core/v1"
)

func performExec() error {
    restConfig, err := config.GetConfig()
    if err != nil {
        return err
    }

    clientset, err := kubernetes.NewForConfig(restConfig)
    if err != nil {
        return err
    }

    cmd := []string{
        "sh",
        "-c",
        "pwd",
    }

    // getting a pod for exec operation (execPod) here...
    execPodName := execPod.Name
    log.Printf("Running exec command in pod %s ...\n", execPodName)

    req := clientset.CoreV1().RESTClient().Post().Resource("pods").Name(execPodName).
        Namespace("default").SubResource("exec")
    option := &v1.PodExecOptions{
        Command: cmd,
        Stdin:   true,
        Stdout:  true,
        Stderr:  true,
        TTY:     true,
    }
    req.VersionedParams(
        option,
        scheme.ParameterCodec,
    )

    exec, err := remotecommand.NewSPDYExecutor(restConfig, "POST", req.URL())
    if err != nil {
        return err
    }

    fmt.Println("--- start of exec pod logs ---")
    err = exec.Stream(remotecommand.StreamOptions{
        Stdin:  os.Stdin,
        Stdout: os.Stdout,
        Stderr: os.Stderr,
    })
    if err != nil {
        return err
    }
    fmt.Println("--- end of exec pod logs ---")

    return nil
}

Here is the RBAC that I applied to both clusters:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  name: manager-role
rules:
- apiGroups:
  - ""
  resources:
  - pods/exec
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch

What did I miss? How to make this work regardless it's running on Kind or not?

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
Zulhilmi Zainudin
  • 9,017
  • 12
  • 62
  • 98
  • `exec` does not run in own pod, it's an API call. That log output will be printed in the operator's manager pod, not the target. – coderanger Jan 01 '21 at 22:41
  • I understood that. The logs above were from operator's manager pod. – Zulhilmi Zainudin Jan 01 '21 at 22:44
  • Sounds like your operator just isn't running correctly then? Also pleeeeeeease rethink your code, you should not use the exec APIs like this. It's unsafe, unscalable, and unmaintainable. – coderanger Jan 01 '21 at 22:59
  • @coderanger - I know using exec operation like this in operator is anti-pattern. I just need it for a small part in my operator during this v1alpha1. Not everything in my operator is exec-driven. It's very unfortunate that I need to rely on exec for this particular part until we finish building the perfect no-exec solution. – Zulhilmi Zainudin Jan 02 '21 at 11:04

0 Answers0