Read a properties file from HDFS

Question

I'm trying to read a Java properties file that is on HDFS like this:

try {
    properties.load(new FileInputStream("hdfs://user/hdfs/my_props.properties"));
} catch (IOException e) {
    throw new RuntimeException("Properties file not found.");
}

But it doesn't seem to work and I get the "Properties file not found." exception. If I replace the path to a local file, it works fine and I'm able to read the file.

Is it possible to read a HDFS file using FileInputStream?

Thanks!

score 2 · Accepted Answer · edited May 23 '17 at 12:06

2

I hope you need to use hadoop jars and also need the FileSystem to read from HDFS. Something like this should be put before your code.

Path pt=new Path("hdfs://user/hdfs/my_props.properties");
FileSystem fs = FileSystem.get(new Configuration());

Refer to: FileInputStream for a generic file System

edited May 23 '17 at 12:06

Community

1
1

answered Mar 10 '15 at 07:00

Harbinger

762
2
14
36

I get "Wrong FS: hdfs://localhost:9000/user/hdfs/my_props.properties, expected: hdfs://nameservice" exception. Any ideas how to set it the path correctly? – user2727704 Mar 10 '15 at 18:57
Worked after passing the namenode:8020/path/to/file. Thanks. – user2727704 Mar 10 '15 at 19:37

score 1 · Answer 2 · answered Mar 04 '21 at 05:29

1

 val fs = FileSystem.get(new Configuration)
 val hdfsPath = new Path("hdfs://user/hdfs/my_props.properties")
 val fis = new InputStreamReader(fs.open(hdfsPath))
 prop.load(fis) // This will be your properties object

answered Mar 04 '21 at 05:29

Raptor0009

258
4
14

Read a properties file from HDFS

2 Answers2