After running this command
pyspark --packages io.delta:delta-core_2.12:2.1.0,org.apache.spark:spark-sql-kafka-0-10_2.12:3.2.1,org.apache.kafka:kafka-clients:2.8.1 --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog" --driver-memory 100g --jars file:///C:/hadoop/jars/hadoop-azure-3.3.2.jar,file:///C:/hadoop/jars/wildfly-openssl-java-2.2.1.Final
I received the following error:
:::: ERRORS
Server access error at url https://repo1.maven.org/maven2/io/delta/delta-core_2.12/2.1.0/delta-core_2.12-2.1.0.pom
(javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake)
Server access error at url https://repo1.maven.org/maven2/io/delta/delta-core_2.12/2.1.0/delta-core_2.12-2.1.0.jar
(javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake)
Server access error at url https://repos.spark-packages.org/io/delta/delta-core_2.12/2.1.0/delta-core_2.12-2.1.0.pom
(javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake)
Server access error at url https://repos.spark-packages.org/io/delta/delta-core_2.12/2.1.0/delta-core_2.12-2.1.0.jar
(javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake)
Maybe interesting for you: I can open URLs in my browser.
As I have read in some posts. The reason could be the TLS version mismatch. What should I do to resolve this issue? I will be really appreciated it if you can show me a step-to-step setting
UPDATE
The
- server ("https://repo1.maven.org/maven2/io") uses TLS1.2
- I removed TLS1.2 and TLS1.1 from
jdk.tls.disabledAlgorithms
in thejava.security
file`
But still receive the handshake problem