3

If I want to ingest logs which are present in HDFS into Filebeat, how can i do that? I can specify any directory that will be on local drives but i want the Filebeat to pick data from HDFS. Is there any way this can be done? Any help will be greatly appreciated.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
saurabht
  • 51
  • 2
  • 11
  • 1
    Filebeat only reads files from local file system. One option may be using **HDFS NFS Gateway.** By this HDFS file system can be mounted as part of the client's local file system. [link](https://hadoop.apache.org/docs/r2.6.4/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html) – Kadir Jan 27 '17 at 13:30

1 Answers1

0

You can mount HDFS to local file system, follow instructions from this answer.

Vishrant
  • 15,456
  • 11
  • 71
  • 120