I need to load com.csv spark packages dynamically to my application, using spark submit , it works
spark-submit --class "DataLoaderApp" --master yarn
--deploy-mode client
--packages com.databricks:spark-csv_2.11:1.4.0 target/scala-2.10/ra-etl_2.10-1.0.0.jar LOAD GGSN /data-sources/DXE_Ver/1_4/MTN_Abuja/GGSN/20160221/GGSN_0_20160221074731.dat
but when I use
spark-submit --class "DataLoaderApp" --master yarn
--deploy-mode client
target/scala-2.10/ra-etl_2.10-1.0.0.jar LOAD GGSN /data-sources/DXE_Ver/1_4/MTN_Abuja/GGSN/20160221/GGSN_0_20160221074731.dat
for below configuration it doesn't work...
val conf = new SparkConf()
.setAppName("Data Loader Application")
.set("spark.jar.packages","com.databricks:spark-csv_2.11:1.4.0")