How to switch kubernetes contexts dynamically with client-go?

Question

I'm building a CLI application that would allow me to run an arbitrary command in my shell against any kube cluster in my kubeconfig that matches a given regex. I want to use the official client-go package to accomplish this, but for some reason, switching kube contexts is less than intuitive. So I'm starting by modifying the example out-of-cluster program in the repo, and I'm struggling with just switching the context to the one I specify. Here is the code I started with, which gets the number of pods in the cluster loaded in the kubeconfig:

package main

import (
    "context"
    "flag"
    "fmt"
    "path/filepath"

    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/util/homedir"
)

func main() {
    var kubeconfig *string
    if home := homedir.HomeDir(); home != "" {
        kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
    } else {
        kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
    }
    flag.Parse()

    // use the current context in kubeconfig
    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        panic(err.Error())
    }

    // create the clientset
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        panic(err.Error())
    }

    pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
    if err != nil {
        panic(err.Error())
    }
    fmt.Printf("There are %d pods in the test cluster\n", len(pods.Items))

}

Unfortunately, I cannot figure out how to load a specific cluster with a name as defined in my kubeconfig. I would love to have a sort of SwitchContext("cluster-name") function, but the number of Configs, ClientConfigs, RawConfigs, and restclient.Configs are confusing me. Any help would be appreciated!

System: Ubuntu 22.04, Intel, kube server version 1.23.8-gke.1900, client version 1.25.3

Have you tried this: https://github.com/kubernetes/client-go/issues/192#issuecomment-349564767 — Rakesh Gupta, Nov 22 '22 at 20:13
Yes, however BuildConfigFromFlags https://pkg.go.dev/k8s.io/client-go@v0.25.4/tools/clientcmd#BuildConfigFromFlags is now builtin and does not do those things. That issue is from more than 5 years ago, and with the state of kube dev, is out of date. — Matanya, Nov 22 '22 at 20:23

score 3 · Accepted Answer · answered Nov 23 '22 at 06:58

You can override the current context via NewNonInteractiveDeferredLoadingClientConfig method from clientcmd package.

package main

import (
    "context"
    "flag"
    "fmt"
    "k8s.io/client-go/rest"
    "path/filepath"

    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/util/homedir"
)

func main() {
    var kubeconfig *string
    if home := homedir.HomeDir(); home != "" {
        kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
    } else {
        kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
    }
    flag.Parse()

    // use the current context in kubeconfig
    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        panic(err.Error())
    }

    // using `contextName` context in kubeConfig
    contextName := "gce"
    config, err = buildConfigWithContextFromFlags(contextName, *kubeconfig)
    if err != nil {
        panic(err)
    }

    // create the clientset
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        panic(err.Error())
    }

    pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
    if err != nil {
        panic(err.Error())
    }
    fmt.Printf("There are %d pods in the test cluster\n", len(pods.Items))

}

func buildConfigWithContextFromFlags(context string, kubeconfigPath string) (*rest.Config, error) {
    return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
        &clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath},
        &clientcmd.ConfigOverrides{
            CurrentContext: context,
        }).ClientConfig()
}

How to switch kubernetes contexts dynamically with client-go?

1 Answers1