0

I have put my file at /hadoop/yarn/local/usercache/root/test_abspath and want to read first line using my UDF. When I ran it using select test('ABCD','ABCD'); I could read the file but when I tried it using Create table as test_tb select test(name,'ABCD') from another_table it fails. in second case f1.exist() was true but f1.canRead() was false I tried changin read write permission by chmod 777, it didn't worked.

Java code to read file is as follows:

public class config {
  public static final String confFilePath = "/hadoop/yarn/local/usercache/root/test_file
  public static void init(){    
        BufferedReader br;
       try {    
            File f1 = new File(confFilePath);
            if (f1.exists())
            {   if (f1.canRead()){

                br = new BufferedReader( new FileReader( confFilePath ));
                String line;
                line = br.readLine();
                str = line;
                logger.info("inside init");
                }
                else{
                    str = "can't read";
                }
            }

In case of select test('ABCD','ACBD') it reads first line of test_file successfully and in case of Create table as test_tb select test(name,'ABCD') from another_table it return can't read String.

Edit: If this is about adding serde.jar then please redirct me to correct jar to download. HDP sandbox version 2.6.5

TheBeginner
  • 405
  • 5
  • 23
  • Is the file you want to read local or on HDFS? your `select` query might work because it's not running on cluster hence Hive has no problems with finding the file, while `create` query is running on cluster. Try to add the file to distributed cache with `add file`, then read it with Java code. Keep in mind that you need to change the file location as it's in the root folder, e.g. `./test_file` – serge_k Sep 04 '18 at 05:54
  • can you please explain more on " 'create' runs on cluster" . I observe that when I tried to get working directory of select its same as my current directory whereas for Create query its /hadoop/yarn/local/usercache/root. Can you please tell the reason behind this? thanks – TheBeginner Sep 04 '18 at 12:58
  • `/hadoop/yarn/local/usercache/root` is it your local machine path or HDFS path? – serge_k Sep 04 '18 at 13:05
  • Its a local machine path. – TheBeginner Sep 04 '18 at 14:23
  • Do you have HDFS in you sandbox? – serge_k Sep 04 '18 at 14:27
  • Yes, I do have it – TheBeginner Sep 04 '18 at 14:31
  • Try to put your file to some place on HDFS and indicate the full path in your code, e.g. `hdfs://path/to/file` – serge_k Sep 05 '18 at 08:29

0 Answers0