I want to use external jar in all instances/nodes of an EMR cluster so that it can be used further in EMR jupyter notebook. I am currently using the follow
#!/bin/bash aws s3 cp s3://<bucket-name>/<prefix>/jars/TFSReconArtifacts-1.0-WithDependencies.jar /home/hadoop/jars/
But while referring the same in EMR jupyter notebook I am not able to access or find anything in this path.(/home/hadoop/jars)
AIM My main aim is to use external jar in EMR jupyter notebook which is bigger in size about 300Mb.I have already tried smaller jars and it works by using this command in jupyter notebook, but for bigger jars it is not working :
%%configure
{ "conf": {
"spark.jars":"s3://<bucketname>/jars/TFSReconArtifacts-1.0-WithDependencies.jar"
}}