Spark submit not working with protected nexus repository

Question

Trying resolve builds from nexus not working if the nexus repo is password protected. giving --repositories http://:<url_safe_password>@:/<repo_path> doesn't work

Answer 1

I was facing this issue, and couldn't find a simple solution, so here it goes:

Resolving password protected repository fails (atleast for http nexus url). to resolve this we can do below things.

• create a ivysetting.xml with your credentials and resolving details (details can be found here)
• pass ivysetting.xml to spark-submit using --conf spark.jars.ivySettings=2/ivysettings.xml

Sample ivysetting.xml for only setting up your nexus repo resolution (this will break local, cached repository resolving)

<ivysettings>
  <settings defaultResolver="nexus"/>
  <property name="nexus-public" value="http://nexus_url:nexus_port/path/to/repo/"/>
  <credentials host="nexus_url_without_http_or_port" realm="Sonatype Nexus Repository Manager" username="user" passwd="secret"/>
<resolvers>
    <ibiblio name="nexus" m2compatible="true" root="${nexus-public}"/>
</resolvers>

</ivysettings>