Access Kubernetes API from a pod in C#

Question

I'm looking for a lightweight way to access the Kubernetes API from a Pod in a C# app.

Kubernetes docs mention two ways of accessing the API from a Pod:

Run kubectl proxy in a sidecar container in the pod, or as a background process within the container

This generally works, and allows for easily hitting an API endpoint with just a line of code or two - example:

using System;
using System.Net;

namespace GetIngresses
{
    class Program
    {
        static void Main(string[] args)
        {
            var apiBaseUrl = "http://127.0.0.1:8001"; // requires kubectl proxy
            Console.WriteLine((new WebClient()).
             DownloadString($"{apiBaseUrl}/apis/networking.k8s.io/v1/ingresses"));
        }
    }
}

However, now there's a running kubectl proxy process to monitor, maintain etc. - this doesn't seem ideal for production.

Use the Go client library, and create a client using the rest.InClusterConfig() and kubernetes.NewForConfig() functions. They handle locating and authenticating to the apiserver.

My app is written in C#, not Go. There's a C# client library which presumably might be able to achieve the same thing. But do I really have to bring a whole client library on board just for a simple GET to a single endpoint?

Ideally, I'd like to just use WebClient, like in the example above. Documentation mentions that

The recommended way to locate the apiserver within the pod is with the kubernetes.default.svc DNS name, which resolves to a Service IP which in turn will be routed to an apiserver.

So, in the example above, can I just do this...

var apiBaseUrl = "http://kubernetes.default.svc"

... and get WebClient to pass the required service account credentials? If yes, how?

The ApplicationInsights.Kubernetes library has an example of what you're after (using HttpClient). You need to weed through some of the DI stuff and filter out the windows vs Linux logic. But it can all be boiled down to two smallish classes (plus entity classes for deserializing the response). Though it does rely on the user having proper Role bindings and it *might* be specific to AKS --- but it could be a place to start — pinkfloydx33, Feb 07 '21 at 21:34
@pinkfloydx33 [this](https://github.com/microsoft/ApplicationInsights-Kubernetes/tree/develop/src/ApplicationInsights.Kubernetes/K8sHttpClient), right? Very helpful, thanks! — Max, Feb 07 '21 at 21:47
Yes that. I actually forked it and turned it into a Serilog enricher after gutting all of the DI and Windows stuff, which is how I know about it — pinkfloydx33, Feb 07 '21 at 22:06
FYI They have the logic for handling the crt and token using the default paths but unfortunately that makes it difficult to run locally (though they do manage that in their tests) — pinkfloydx33, Feb 07 '21 at 22:24

Jonas · Accepted Answer · 2021-02-07T22:30:02.317

Ideally, I'd like to just use WebClient

The Kubernetes is a REST API, so this would work. As shown on Directly accessing the REST API using kubectl proxy it is easy to explore the API using e.g. curl.

Example with curl and kubectl proxy - response is in json format.

curl http://localhost:8080/api/v1/pods

The complicating factor is that you probably need a private certificate bundle, and it is good practice to properly validate this for security reasons. When accessing the API from a Pod, the client certificate is located on /var/run/secrets/kubernetes.io/serviceaccount/ca.crt and in addition, you need to authenticate using the token located on /var/run/secrets/kubernetes.io/serviceaccount/token

But do I really have to bring a whole client library on board just for a simple GET to a single endpoint?

What you get from a client library is:

Implemented authentication using certificates and tokens
Typed client access - instead of hand code urls and requests

The dotnet-client example shows how the "Typed client access" looks like, for "listing Pods in the default namespace" (see authentication alternatives):

var config = KubernetesClientConfiguration.InClusterConfig()  // auth from Pod
IKubernetes client = new Kubernetes(config);
Console.WriteLine("Starting Request!");

var list = client.ListNamespacedPod("default");
foreach (var item in list.Items)
{
    Console.WriteLine(item.Metadata.Name);
}

Very helpful, thanks! I'm learning here that the auth part isn't quite trivial, hence a client lib is, as you're pointing out, beneficial. — Max, Feb 07 '21 at 22:14
Yes, been exposed to that problem. The client lib can be configured to use your local kubectl-config or use config as from a Pod. I typically implement both in my code, and use a flag to choose - so I both can run locally and from a Pod. — Jonas, Feb 07 '21 at 22:16

Access Kubernetes API from a pod in C#

1 Answers1

Linked