It is known that ISP has to log various network data for various purposes, such as law enforcement needs.
However, what are some examples of software tools used by ISPs to browse this huge amount of network logs?
In addition, how time-consuming is for ISP, for example, to look up for a user behind a given IP address if they have been subpoenaed?
It depends on the kind of the log. But nowadays lot of people use Security Information and Event Management (SIEM) systems.
SIEM solutions provide a holistic view of what is happening on a network in real-time and help IT teams to be more proactive in the fight against security threats.
What is unique about SIEM solutions is that they combine Security Event Management (SEM) - which carries out analysis of event and log data in real-time to provide event correlation, threat monitoring an incident response - with Security Information Management (SIM) which retrieves and analyzes log data and generates a report. For the organization that wants complete visibility and control over what is happening on their network in real-time, SIEM solutions are critical.
SIEM software collects and aggregates log data generated throughout the organization’s technology infrastructure, from host systems and applications to network and security devices such as firewalls and antivirus filters.
The software then identifies and categorizes incidents and events, as well as analyzes them. The software delivers on two main objectives, which are to provide reports on security-related incidents and events, such as successful and failed logins, malware activity and other possible malicious activities and send alerts if analysis shows that an activity runs against predetermined rulesets and thus indicates a potential security issue.
Some free and open source SIEM:
SIEM can use AI (artificial intelligence) to gain predictive capabilities. Often, AI in SIEM manifests as machine learning; this vital capability learns about threats as it acquires threat intelligence and deflects attacks in the field. Machine learning enables easier threat detection across large data sets, alleviating some threat hunting responsibilities from the security team.
