Importing data from HDFS to Hive table

Question

I have my data in data/2011/01/13/0100/file in HDFS, each of thes file contain data in tab separated, say name, ip , url.

I want to create a table in Hive and import the data from hdfs, table should contain time,name, ip and url.

How can I import these using Hive ? r the data should be in some other format so that I can import the time as well ?

Check this question: http://stackoverflow.com/questions/9626789/how-can-i-partition-a-table-with-hive — Raul Guiu, Apr 17 '12 at 21:43

score 4 · Answer 1 · answered Apr 28 '11 at 01:35

4

You need to create the table to load the files into and then use the LOAD DATA command to load the files into the Hive tables. See the Hive documentation for the precise syntax to use.

Regards, Jeff

answered Apr 28 '11 at 01:35

Jeff Hammerbacher

4,226
2
29
36

score 0 · Answer 2 · answered Sep 24 '14 at 12:25

You can create an external table for such data.

Something like:

CREATE EXTERNAL TABLE log_data (name STRING, ip STRING, url STRING) PARTITIONED BY (year BIGINT, month BIGINT, day BIGINT, hour BIGINT) row format delimited fields terminated by '\t' stored as TEXTFILE location 'data'

score 0 · Accepted Answer · edited May 23 '17 at 11:51

0

To do this you have to use partitions, read more about them here:

edited May 23 '17 at 11:51

Community

1
1

answered Apr 23 '11 at 07:59

wlk

5,695
6
54
72

Importing data from HDFS to Hive table

3 Answers3

Linked