We have a spark only cluster for multiple users, in spark-default.conf have below property set
spark.driver.memory 2g
spark.executor.cores 1
spark.executor.memory 2g
As we have a multiple users, i dont want users to pass below parameters in spark-submit command as this can utilise complete cluster resources.
--driver-memory 5g
--executor-cores 5
--executor-memory 5g
is that any possible way to restrict passing above runtime value in spark-submit command ?
try added this property as max value but still users are running with some random values less than 5
spark.cores.max 5
