1

I'm trying to add my custom scheduler with scheduler plugin in Kubernetes. Due to no detailed example in Kubernetes document (scheduling-framework, 20180409-scheduling-framework)

So I simply made it for testing. But, I got error: unable to recognize "test.yaml": no matches for kind "KubeSchedulerConfiguration" in version "kubescheduler.config.k8s.io/v1alpha1".

How could I configure and use scheduler plugin?

In my case, test configurations are

apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
plugins:
  queueSort:
    enabled:
    - name: test

pluginConfig:
- name: test-

kubectl command

$ kubectl apply -f test.yaml
error: unable to recognize "test.yaml": no matches for kind "KubeSchedulerConfiguration" in version "kubescheduler.config.k8s.io/v1alpha1"

When I created as pod,

$ kubectl create -f test.yaml
$ kubectl logs pod test
E0920 15:55:47.970707       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 15:55:47.971808       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:48.962810       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "nodes" in API group "" at the cluster scope
E0920 15:55:48.963476       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:48.964615       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 15:55:48.965674       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 15:55:48.966800       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 15:55:48.967967       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "services" in API group "" at the cluster scope
E0920 15:55:48.969030       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "pods" in API group "" at the cluster scope
E0920 15:55:48.970095       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0920 15:55:48.971150       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0920 15:55:48.972252       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 15:55:48.973396       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:49.964805       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "nodes" in API group "" at the cluster scope
E0920 15:55:49.965338       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:49.966313       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 15:55:49.967467       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 15:55:49.968448       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 15:55:49.969534       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "services" in API group "" at the cluster scope
E0920 15:55:49.970775       1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Pod: pods is f^C

Enviroment

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

go

$ go version
go version go1.13 linux/amd64

go module

$ cat go.sum
...

k8s.io/kubernetes v1.16.0 h1:WPaqle2JWogVzLxhN6IK67u62IHKKrtYF7MS4FVR4/E=
k8s.io/kube-scheduler v0.0.0-20190918162820-3b5c1246eb18 h1:WnbesbtFrWlZ8wwgDuP1KN+6E+XLDGzG4ztojMxfHsc=

...
yotion
  • 13
  • 1
  • 3

2 Answers2

0

The scheduler configuration is not applied to the API server with kubectl apply command, it is passed to the scheduler process with kube-scheduler --config=/path/to/config. Read more about it on Github.

--config is the entry where you should specify which parameters the scheduler will take. From the API’s perspective, that config file is supposed to contain a KubeSchedulerConfiguration object:

 # content of the file passed to "--config"
 apiVersion: kubescheduler.config.k8s.io/v1alpha1
 kind: KubeSchedulerConfiguration
 clientConnection:
   kubeconfig: "/var/run/kubernetes/scheduler.kubeconfig"
 algorithmSource:
   policy:
     file:
       path: "/root/config/scheduler-extender-policy.json"

Refer to this article, which describe how to create custom Kubernetes scheduler and configuration of Scheduler.

aga
  • 3,790
  • 3
  • 11
  • 18
  • Thanks!! I could guess what the problem was – yotion Sep 29 '19 at 10:25
  • I think that @yotion wants to use the scheduling framework. However, the links provided refer to the scheduler extender. – iwita Jan 30 '20 at 18:09
  • @iwita indeed, I am also searching for a way to deploy plugin using scheduling framework API with off-the-tree feature, do you have any tutorial that you know? – repié Feb 19 '20 at 13:35
0

KubeSchedulerConfiguration is a kind of configmap, if you want to apply if you have to command like below -

kubectl create -n kube-system configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml

Here -

kube-system = your target namespace

my-scheduler-config = the name of your config map, it can be any name

/root/my-scheduler-config.yaml = replace with your yaml config file name & location

Kalyan Halder
  • 1,485
  • 24
  • 28
  • 1
    Hi, you are saying that it's a kind of configmap. After creating the configmap, how is it applied to the scheduler? Are you proposing that the scheduler will discover the configmap, regardless of its name, and use it? Could you link to an official documentation page that corroborates the specific information? thanks. – Sam Jan 27 '23 at 23:58