1

We have small hadoop and Greenplum cluster.
Current data pipeline flow is :

External table >> hadoop-hawq external readbale table >>hawq internal table.

Output : 1.WE are trying to extend data pipeline using GREENPLUM. Basically wan to push HAWQ Internal table or external readable table data directly into greenplum. Reason is because we want to edit our file. also, HAWQ does not support Update and delete. is there any alternate way to approach or push the data. Please guide.

2.How to access HDFS data via GPDB external table with gphdfs protocol

Thanks in Advance!

NEO
  • 389
  • 8
  • 31

1 Answers1

2

If you want to push data in HAWQ internal table to Greenplum Database, you can:

1) Unload data in HAWQ internal table to file on HDFS using writable external table. Here is an example for doing the unload: http://gpdb.docs.pivotal.io/4380/admin_guide/load/topics/g-unloading-data-using-a-writable-external-table.html

2) Then load data in HDFS file to Greenplum Database using readable external table with protocol like gphdfs, gpfdist, etc. You can refer to http://gpdb.docs.pivotal.io/4320/admin_guide/load.html for details.

If you want to push data in readable external table in HAWQ to Greenplum Database, you can directly use readable external table in Greenplum Database as in HAWQ.

For gphdfs, here are some example which would help: http://gpdb.docs.pivotal.io/4380/admin_guide/load/topics/g-example-1-greenplum-file-server-gpfdist.html

huor
  • 116
  • 4