1

How to process Azure Log events from Event hub and filter based on criteria.

We are trying to filter specific critical or security related Diagnostic and Activity logs before feeding into our Onprem SIEM solution.

Can someone please guide me how to filter the data from Event hub and then re-ingest into another event hub. Weather this is possible or any other alternatives available out there.

At a high level , the flow is shown below. Source from Azure Activity or Diagnostic Logs (Monitor) -> Event Hub -> Filter/Query -> Event Hub

Azure Event Log Data Flow Diag

Anil Kash
  • 11
  • 1

1 Answers1

1

Event Hub itself can't do any filtering for you, it is just an ingestion tool so you are going to need to look at another tool to look at this data and filter out what you need. The easiest tool for doing this is Stream Analytics, which an can process the data from event hub directly and then send it where you need. See an example here.

Sam Cogan
  • 38,736
  • 6
  • 78
  • 114
  • Thanks for your reply, yes its quite evident from my post that I am not trying to filter any data from EventHub itself , but post it, sorry if it was not clear enough. I have already tried stream analytics, and that is not apt for processing JSON payload from Azure Event Logs. – Anil Kash Apr 22 '20 at 05:43
  • what's wrong with steam analytics? This is what it was designed for – Sam Cogan Apr 22 '20 at 11:15