Getting NullPointerException when using a S3 job file with Samza

Asked Apr 22 '15 at 18:37

Active Apr 23 '15 at 22:38

Viewed 376 times

I'm getting the following exception when passing a S3 file path to the yarn.package.path.

Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:433)
at org.apache.samza.job.yarn.ClientHelper.submitApplication(ClientHelper.scala:111)
at org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:54)
at org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:47)
at org.apache.samza.job.JobRunner.run(JobRunner.scala:62)
at org.apache.samza.job.JobRunner$.main(JobRunner.scala:37)

I'm able to curl the s3 file from the same box (after exporting the AWS environment variables). This is how package path is set in my job properties file:

yarn.package.path=s3n://{ACCESS_KEY}:{SECRET_KEY}@bucketname/path1/path2/tar.gz

asked Apr 22 '15 at 18:37

Joseph

By any chance, does your secret key have a '/' in it? – Jon Bringhurst Apr 24 '15 at 17:22
@JonBringhurst, has a '+', but not a '/' – Joseph Apr 24 '15 at 19:28

Getting NullPointerException when using a S3 job file with Samza

0 Answers0