0

I want to get a notification when a certain file/ directory has been deleted in HDFS. It would also be possible to generate a kind of log file with timestamp, affected file/ directory and so on. I need this due to traceability requirements.

Is there any script, software et cetera to cover this kind of task?

Pigger
  • 11
  • 1
  • You could probably use something like Nagios to monitor the namenode log files. – LiMuBei May 11 '15 at 14:36
  • Do you know where the namenode log files are located? – Pigger May 11 '15 at 15:05
  • Depends on your Hadoop distribution. According to this blog article http://blog.cloudera.com/blog/2009/09/apache-hadoop-log-files-where-to-find-them-in-cdh-and-what-info-they-contain/ at least in a Cloudera dist it should be in /var/log/hadoop/hadoop-hadoop-namenode-... – LiMuBei May 11 '15 at 15:26
  • OK, thank you. But I think there must be an existing software for auditing the HDFS access log files, or? – Pigger May 11 '15 at 15:49

1 Answers1

0

Use HdfsAdmin.getInotifyEventStream(...) to open an DFSInotifyEventInputStream. Monitor the events in the stream for an event of type Event.UnlinkEvent.

DanN
  • 106
  • 1
  • 2