2

How do you load a text file to an Apache Kudu table?

Does the source file need to be in HDFS space first?

If it doesn't share the same hdfs space as other hadoop ecosystem programs (ie/ hive, impala), is there Apache Kudu equivalent of:

hdfs dfs -put /path/to/file

before I try to load the file?

boethius
  • 418
  • 6
  • 15

1 Answers1

3

The file need not to be in HDFS first.It can be taken from an edge node/local machine.Kudu is similar to Hbase.It is a real-time store that supports key-indexed record lookup and mutation but cant store text file directly as in HDFS.For Kudu to store the contents of a text file,it needs to be parsed and tokenised.For that, you need to have Spark execution/java api alongwith Nifi (or Apache Gobblin) to perform the processing and then storing it in Kudu table.

Or

You can integrate it with Impala allowing you to use Impala to insert, query, update, and delete data from Kudu tablets using Impala’s SQL syntax, as an alternative to using the Kudu APIs to build a custom Kudu application.Below are the steps:

  1. Import the file in hdfs
  2. Create an external impala table.
  3. Then insert the data in the table.
  4. Create a kudu table using keyword stored as KUDU and As Select to copy the contents from impala to kudu.

In this link you can refer for more info- https://kudu.apache.org/docs/quickstart.html

Subash
  • 887
  • 1
  • 8
  • 19
  • Thanks for the link to the docs - I will test this out, but given everything I've read, I think it should work! – boethius Dec 12 '17 at 18:50