I am getting the below error even though I am running as administrator
C:\Spark\spark-3.3.1-bin-hadoop3\bin>
C:\Spark\spark-3.3.1-bin-hadoop3\bin>spark-shell --packages io.delta:delta-core_2.12:1.2.1,org.apache.hadoop:hadoop-aws:3.3.1 --conf spark.hadoop.fs.s3a.access.key=<my key> --conf spark.hadoop.fs.s3a.secret.key=<my secret> --conf "spark.hadoop.fs.s3a.endpoint=<my endpoint> --conf "spark.databricks.delta.retentionDurationCheck.enabled=false" --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog"
"C:\Program Files\Eclipse Adoptium\jdk-11.0.13.8-hotspot\bin\java" -cp "C:\Spark\spark-3.2.3-bin-hadoop3.2\bin\..\conf\;C:\Spark\spark-3.2.3-bin-hadoop3.2\jars\*" "-Dscala.usejavacp=true" -Xmx1g org.apache.spark.deploy.SparkSubmit --conf "spark.hadoop.fs.s3a.endpoint=<my endpoint> --conf spark.databricks.delta.retentionDurationCheck.enabled=false --conf spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog > C:\Users\shari\AppData\Local\Temp\spark-class-launcher-output-8183.txt" --conf "spark.hadoop.fs.s3a.secret.key=<my secret>" --conf "spark.hadoop.fs.s3a.access.key=<my key>" --class org.apache.spark.repl.Main --name "Spark shell" --packages "io.delta:delta-core_2.12:1.2.1,org.apache.hadoop:hadoop-aws:3.3.1" spark-shell
The system cannot find the file C:\Users\shari\AppData\Local\Temp\spark-class-launcher-output-8183.txt.
Could Not Find C:\Users\shari\AppData\Local\Temp\spark-class-launcher-output-8183.txt
However, if I execute just spark-shell it works.
Can anyone please help me with it ?
OS: Windows 11 Spark: Apache Spark 3.3.1 Java: openjdk version "11.0.13" 2021-10-19
thanks
