I'd like to submit a SparkApplication to a Kubernetes cluster programmatically from python.
A job definition job.yaml like this
apiVersion: sparkoperator.k8s.io/v1beta1
kind: SparkApplication
metadata:
name: my-test
namespace: default
spec:
sparkVersion: "2.4.0"
type: Python
...
runs without problems using kubectl apply -f job.yaml, but I cannot figure out whether and how I can use the kubernetes-client to start this job programmatically.
Does anyone know how to do this?
Here is the example mentioned, how to create third party resource on kubernetes using kubernetes python client.
https://github.com/kubernetes-client/python/blob/master/examples/create_thirdparty_resource.md
Hope this helps.
This is probably what you are looking for:
from __future__ import print_function
import time
import kubernetes.client
from kubernetes.client.rest import ApiException
from pprint import pprint
# Configure API key authorization: BearerToken
configuration = kubernetes.client.Configuration()
configuration.api_key['authorization'] = 'YOUR_API_KEY'
# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
# configuration.api_key_prefix['authorization'] = 'Bearer'
# create an instance of the API class
api_instance = kubernetes.client.CustomObjectsApi(kubernetes.client.ApiClient(configuration))
group = 'group_example' # str | The custom resource's group name
version = 'version_example' # str | The custom resource's version
namespace = 'namespace_example' # str | The custom resource's namespace
plural = 'plural_example' # str | The custom resource's plural name. For TPRs this would be lowercase plural kind.
body = NULL # object | The JSON schema of the Resource to create.
pretty = 'pretty_example' # str | If 'true', then the output is pretty printed. (optional)
try:
api_response = api_instance.create_namespaced_custom_object(group, version, namespace, plural, body, pretty=pretty)
pprint(api_response)
except ApiException as e:
print("Exception when calling CustomObjectsApi->create_namespaced_custom_object: %s\n" % e)
