We are currently running Spark jobs on EKS 1.24. The jobs run in a PSS restricted namespace. We specify the securityContext settings in a pod template file. These settings provide a seccompProfile which is required by the PSS restricted policy. However, we discovered that the setting is not being added by Spark but by a PSP annotation. Since PSP is not available in EKS 1.25 this prevents us from upgrading without relaxing the PSS policy. It would appear that Spark has a bug in applying the securityContext settings in pod template files.
We switched the PSP from a restricted one to a privileged one. The restricted one has an annotation to specify the seccompProfile.type to be RuntimeDefault while the privileged policy did not have this annotation. We assumed that using the privileged PSP would have no effect, but when we made the switch Spark jobs began to fail. We realized that the seccompProfile was not being set because it is in the pod template file, but because PSP is inserting the seccompProfile setting with a mutating webhook. The expectation is that all the settings in the pod template file are applied without PSPs.
Has anyone been able to deploy a Spark job in a PSS restricted namespace hosted on a 1.25 Kubernetes cluster?
