0

We are running spark job which connect to oracle and fetch some data. Always attempt 0 or 1 of JDBCRDD task fails with below error. In subsequent attempt task get completed. As suggested in few portal we even tried with -Djava.security.egd=file:///dev/urandom java option but it didn't solved the problem. Can someone please help us in fixing this issue.

ava.sql.SQLRecoverableException: IO Error: Connection reset by peer, Authentication lapse 59937 ms.
    at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:794)
    at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:688)
Rishi Saraf
  • 1,644
  • 2
  • 14
  • 27

3 Answers3

1

Issue was with java.security.egd only. Setting it through command line i.e -Djava.security.egd=file:///dev/urandom was not working so I set it through system.setproperty with in job. After that job is no more giving SQLRecoverableException

Rishi Saraf
  • 1,644
  • 2
  • 14
  • 27
  • The above solution also resolves the issue of sqlcl not connecting to Oracle and throwing an authentication lapse error. To set it through the command line you actually need -Djava.security.egd=file:/dev/./urandom due to a jdk bug – Damian Mar 09 '18 at 14:53
0

This Exception nothing to do with Apache Spark ,"SQLRecoverableException: IO Error:" is simply the Oracle JDBC driver reporting that it's connection to the DBMS was closed out from under it while in use. The real porblem is at the DBMS, such as if the session died abruptly. Please check DBMS error log and share with question.

Similer problem you can find here https://access.redhat.com/solutions/28436

vaquar khan
  • 10,864
  • 5
  • 72
  • 96
0

Fastest way is export spark system variable SPARK_SUBMIT_OPTS before running your job.

like this: export SPARK_SUBMIT_OPTS=-Djava.security.egd=file:dev/urandom I'm using docker, so for me full command is:

docker exec -it spark-master 
bash -c "export SPARK_SUBMIT_OPTS=-Djava.security.egd=file:dev/urandom && 
/spark/bin/spark-submit --verbose --master spark://172.16.9.213:7077 /scala/sparkjob/target/scala-2.11/sparkjob-assembly-0.1.jar"
  1. export variable
  2. submit job
arbuzov
  • 863
  • 6
  • 11