How to read subdirectory data into bigsql table?

Question

Bigsql is not able to read data from subdirectory like hive can by setting parameters.

set tblproperties (
    "hive.input.dir.recursive" = "TRUE",
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE",
    "mapred.input.dir.recursive" = "TRUE")

I tried adding above parameters into bigsql tableproperties but it's not able to read subdirectory data.

What parameters I need to set in bigsql to read subdirectory data?

score 0 · Answer 1 · answered Apr 10 '19 at 14:25

As per my understanding, Bigsql only looks up for files in parent directory. It doesn't even bother checking sub-directories. It will display an empty table because it cannot read the data recursively.This feature is still under product improvement ideas with IBM product and engineering team.

How to read subdirectory data into bigsql table?

1 Answers1