0

I have a requirement to create a dynamic file based on the content in hadoop job.properties and then put it in Distributed Cache.

When I create the file I see that it has been created with the path of "/tmp".

I create a symbolic name and refer to this file in the cache. Now, when I try to read the file in the Dis. cache I am not able to access it. I get th error caused by: java.io.FileNotFoundException: Requested file /tmp/myfile6425152127496245866.txt does not exist.

Can you please let me know If should I need to specify the path also while creating the file and also use that path while accessing/reading the file.

I only need the file to be available only till the job is running.

javauser
  • 91
  • 4
  • 11

1 Answers1

0

I don't really get your meaning of

I only need the file to be available only till the job is running

But, when I practice to use distributed cache , I use path like this : 

final String NAME_NODE = "hdfs://sandbox.hortonworks.com:8020";
job.addCacheFile(new URI(NAME_NODE + "/user/hue/users/users.dat"));

hope this will help you .

G_cy
  • 994
  • 3
  • 13
  • 28
  • Thanks for your response. What I meant is I would like the file to be available in the cache only till the job is running. Since the file is based on content in job.properties which will change with every job run. My query is on the fact that when I created the file dynamically the absolute path of the file was '/tmp/myfile6425152127496245866.txt' . But, once I put it in the cache and later try to refer it, I get '/tmp/myfile6425152127496245866.txt' does not exist. Could it be that it is referring the file in a different folder whereas it was created somewhere else..? – javauser Mar 31 '15 at 17:22
  • hmm, I think there should be some problem with the path you use. In my case, if I omit the NAME_NODE, I will fail,too. but i don't know whether it is the same situation in your case. – G_cy Mar 31 '15 at 17:57