Accessing AKS kubeconfig file from go program

Question

I am trying to write a Go program to get pod logs from the cluster. I am using AKS kubernetes cluster. How can I access the kubeconfig file inside my script? Following is my code:

package main

import (
    "context"
    "flag"
    "fmt"
    "time"
    "os"
    "path/filepath"

    "k8s.io/apimachinery/pkg/api/errors"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/kubernetes"
    //"k8s.io/client-go/rest"
    "k8s.io/client-go/tools/clientcmd"
    //
    // Uncomment to load all auth plugins
    // _ "k8s.io/client-go/plugin/pkg/client/auth"
    //
    // Or uncomment to load specific auth plugins
    // _ "k8s.io/client-go/plugin/pkg/client/auth/azure"
    // _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
    // _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
    // _ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
)

func main() {
    /*// creates the in-cluster config
    config, err := rest.InClusterConfig()
    if err != nil {
        panic(err.Error())
    }*/
    
    fmt.Printf("Creating cluster config")
    
    kubePtr := flag.Bool("use-kubeconfig", false, "use kubeconfig on local system")
    flag.Parse()
    
    fmt.Printf("Updating the existing config")

    var kubeconfig string

    if *kubePtr == true {
        kubeconfig = filepath.Join(os.Getenv("HOME"), ".kube", "config")
    } else {
        kubeconfig = ""
    }
    
    fmt.Printf("Building config from flags")

    config, err := clientcmd.BuildConfigFromKubeconfigGetter("", kubeconfig)
    
    fmt.Printf("creating the clientset")
    
    
    // creates the clientset
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        panic(err.Error())
    }
    for {
        // get pods in all the namespaces by omitting namespace
        // Or specify namespace to get pods in particular namespace
        pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
        if err != nil {
            panic(err.Error())
        }
        fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))

        // Examples for error handling:
        // - Use helper functions e.g. errors.IsNotFound()
        // - And/or cast to StatusError and use its properties like e.g. ErrStatus.Message
        _, err = clientset.CoreV1().Pods("default").Get(context.TODO(), "example-xxxxx", metav1.GetOptions{})
        if errors.IsNotFound(err) {
            fmt.Printf("Pod example-xxxxx not found in default namespace\n")
        } else if statusError, isStatus := err.(*errors.StatusError); isStatus {
            fmt.Printf("Error getting pod %v\n", statusError.ErrStatus.Message)
        } else if err != nil {
            panic(err.Error())
        } else {
            fmt.Printf("Found example-xxxxx pod in default namespace\n")
        }

        time.Sleep(10 * time.Second)
    }
}

I am getting error in line 51. Following is my error:

error creating inClusterConfig, falling back to default config: unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined

Where do I find KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT and how do I pass it? I could not find any example.

Answer 1

From the example here the code should be like below. Notice the usage of BuildConfigFromFlags function to load the kubeconfig file.

func main() {
    var kubeconfig *string
    if home := homedir.HomeDir(); home != "" {
        kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
    } else {
        kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
    }
    flag.Parse()

    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        panic(err)
    }
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        panic(err)
    }
    ...

Make sure that there is a valid kubeconfig file at ~/.kube/config location. On AKS you can run below command to get the kubeconfig file which puts the file at ~/.kube/config

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

Answer 2

First thing I noticed is that you didn't mention how you are starting your program.

Looking into the code I saw you are creating a kubePtr flag that defaults to false. When flag is set to true it sets kubeconfig variable to kubeconfig path, but beacuse it is false (by default) it sets it to "" and that's why it cant find the config.

---

After you set this flag to true you saw the following error:

cannot use kubeconfig (type string) as type clientcmd.KubeconfigGetter in argument to clientcmd.BuildConfigFromKubeconfigGetter

whihc means that you have type mismatch. Let's have a look at BuildConfigFromKubeconfigGetter() function argument types:

func BuildConfigFromKubeconfigGetter(masterUrl string, kubeconfigGetter KubeconfigGetter) (*restclient.Config, error)

Notice that you are passing a string as an argument that is expected to be of type KubeconfigGetter.

Preferably use different function like clientcmd.BuildConfigFromFlags(), that as an argument is expecting a path (string) to kubeconfig file.

In official client-go library repository on github you can find several examples that can help you get started with clien-go library.

E.g. take a look at this official example, and notice how the client is configured.

Answer 3

I use this command:

go test ./test/e2e -kubernetes-config ~/.kube/config

-kubernetes-config is the param where you specify the path to the kubeconfig file where you are authenticated against.

If Go test returns exit 1 code, retry test and check if resources are being created and debug if for example pod failed due to ErrImagePull and other events.