4

If yes please guide me basic steps for this. I scorll through pages and realized that its possible to do it using core-site.xml

sparkingmyself
  • 148
  • 2
  • 15

1 Answers1

2

You can easily transferring files to HDFS File System by using PutHDFS Processor.

Look at this below Link.,

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.PutHDFS/

In this processor you have to specify following properties is enough to transfer files into HDFS.

1.Hadoop Configuration Resources: -A file or comma separated list of files

/Hadoop/conf/core-site.xml,/Hadoop/conf/hdfs-site.xml,/Hadoop/conf/yarn-site.xml,/Hadoop/conf/mapred-site.xml

2.Directory: Specify location of your HDFS Directory

/FilesFromNiFi/

3.Conflict Strategy:(append/replace/ignore/fail)

append

For example:

If u need to move the file from local drive into HDFS means using below NiFi workflow

GetFile-->PutHDFS

GetFile-->Just give your input directory.

putHDFS-->Specify your HDFS location

You need to drag a processor into UI with respect to below image.,

enter image description here

Refer this link to drag this two processors GetFile and putHDFS into NiFi UI

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#adding-components-to-the-canvas

http://zdatainc.com/2016/02/hello-nifi-data-orchestration-using-hortonworks-dataflow-hdf/

let me know if you face any problem.

Mister X
  • 3,406
  • 3
  • 31
  • 72
  • I am unable to locate above components(GetFile PutHDFS) in Apache NiFi? some settings to include that? – sparkingmyself Feb 10 '17 at 12:52
  • I have edited my answer.You have to drag those two processor into UI with respect to Link attached in answer – Mister X Feb 11 '17 at 04:01
  • 1
    Thanks you for your reply. My NiFi binary was corrupted and I was seesing only 7 processor as an option to select among. I am corrected above issue and now its working fine and I am able to see 183 odd processor in the option. – sparkingmyself Feb 11 '17 at 16:59