Questions tagged [nxlog]

NXLOG is a universal log collector and forwarder supporting different platforms, log sources and protocols.

NXLog can work in a heterogenous environment collecting event logs from thousands of different sources in many formats. NXLog can accept event logs from TCP, UDP, file-based logs, database and various other sources in different formats such as Syslog, Windows Event Log, DNS debug log etc.

It can perform log rewrite, correlation, alerting, pattern matching, execute scheduled jobs, log rotation and log conversion in other formats such as to JSON. It was designed to be able to fully utilize today's multi-core CPU systems. Its multi-threaded architecture enables input, log processing and output tasks to be executed in parallel. Using a high-performance I/O layer, it is capable of handling thousands of simultaneous client connections and process log volumes above the 100.000 EPS range.

NXLog tries hard to minimize losing log messages. It does not drop any unless instructed to. It can process input sources in a prioritized order, meaning that a higher priority source will be always processed before others. This can further help avoiding UDP message loss (as an example). In case of network congestion or other log transmission problems, NXLog can buffer messages on the disk or in memory. Using loadable modules it supports different input sources and log formats, not only limited to Syslog but Windows Event Log, audit logs or even custom binary application logs.

It is possible to further extend its functionality by using custom loadable modules similarly to the Apache Web server. In addition to the online log processing mode, it can be used to process logs in batch mode in an offline fashion. A powerful configuration language with an Apache style configuration file syntax enables it to rewrite logs, send alerts, execute external scripts or do virtually anything based on any criteria specified using the NXLog configuration language.

Please see the NXLog User Guide for more information.

91 questions
0
votes
0 answers

issue with nxlog - logs arrived tagged as USER.NOTICE

First question here... I'm struggling with nxlog that behaves strangely: some logs doesn't enter my rsyslog fromhost-ip filtering rules when other work neatly, with the exact same config file, and fall down on the user.log file... For what I have…
dkdlv
  • 1
  • 1
0
votes
1 answer

Modify raw_event to send custom logs using NXLog

I would like to modify the IIS logs for further transfer to the destination. Now I am parsing the IIS log with the xm_csv module, as in the template. UndefValue is disabled to not get empty. How can I interact with parsed data from w3c_parser? For…
0
votes
1 answer

Unable to parse the Test logs in windows machine using Nxlog

I wanted to process text logs and parse to json format using nxlog, but i dont find any document for configuring such custom logs in Nxlog- userguide, as i could see only event/system logs processing documents only available. Can someone help me…
0
votes
1 answer

NXLog Forwarding to CloudWatch possible?

I have found a number of items on collecting logs from CloudWatch for NXLog to then send somewhere else, but none on forwarding events to CloudWatch? It isn't really a destination, but maybe it is possible. The benfit is the SNARE format and it is…
0
votes
1 answer

Nxlog cannot collect logs in certain directories

Background Use nxlog to collect the logs of win server 2012 R2 server. The configuration file is as follows define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define…
ChaosFish
  • 62
  • 1
  • 8
0
votes
1 answer

Delete fields from NXLOG

I am trying to reduce the size of the message sent from my Windows event logs to graylog and I cannot for the life of me figuring out how tell it to drop certain fields The only thing I can figure out is that i'm supposed to use delete() but how to…
giveen
  • 43
  • 1
  • 6
0
votes
1 answer

run bash script when ERROR spotted in logs in nxlog

I am trying to run a script every time an error is found in logs Module xm_exec Module im_file File "/home/rafal/gitprojects/mst-sender/hub.cloudradar-error.log" if…
rafalf
  • 425
  • 7
  • 16
0
votes
1 answer

Regular expression for nxlog

I have a set of logs which do not have a fixed line number. I am trying to get these logs to Elasticsearch through nxlog. ------------------------------------------------------ Timestamp: 2020-03-03 00:05:04Z|User -DEFAULT|Cart…
AlisonGrey
  • 497
  • 1
  • 7
  • 23
0
votes
1 answer

NXLog Windows Events and Source

I want to upload events only from a "puppet" source, which is "Windows Logs\Application". I guess I have to change the line How to filter source "puppet" in nxlog.conf? Module …
Mr. NoNe
  • 53
  • 1
  • 8
0
votes
1 answer

Can you create custom filter modules or functions for nxlog?

We are using nxlog community edition. Is there any way in the conf file to call out to a custom module that filters and modifies messages? I would like to build this in a more sensible language than the one nxlog uses. We also need a little…
mbr_at_ml
  • 3,585
  • 2
  • 13
  • 12
0
votes
1 answer

Reading XEL (SQL Server eXtended Event file) by log collectors such as filebeats or nxlog

I want to create a centralized log with graylog and log collectors such as filebeats or nxlogs. I know Microsoft SQL Server extended events creates .XEL files. but they are binary files. Is there any clean way to read this xel files by filebeats or…
Arash Mousavi
  • 2,110
  • 4
  • 25
  • 47
0
votes
1 answer

NXLog with HTTPS: How to configure SSL and certificate?

I'm trying to use NXLog to post to a service in https. Unfortunately, I keep receiving this error for every post attempts: 2018-12-19 09:11:14 ERROR SSL certificate verification failed: unable to get local issuer certificate (err: 20) The service we…
Jean-François Côté
  • 4,200
  • 11
  • 52
  • 88
0
votes
2 answers

NXLog always forward file on disk (even if it hasn't change)

I have the following NXLog config that reads a file from disk and forwards it to another host (syslog). By default, it appears NXLog only forwards the data in the log file if it has changed. How can you configure 'nxlog.conf' so that it always…
GreekFire
  • 359
  • 4
  • 15
0
votes
1 answer

NXLOG how to merge multilines regex

I have a log like this: 14:40:33.476 [WebContainer : 149] sometihng sometihng 14:40:33.476 [WebContainer : 149] sometihng sometihng 14:40:33.476 [WebContainer : 149] sometihng sometihng 14:40:33.476 [WebContainer : 245] csometihng sometihng…
madi
  • 160
  • 1
  • 13
0
votes
1 answer

Why is Nxlog is using up 100% cpu leaving system unusable?

os: Windows server 2008 R2 with Nxlog Enterprise 4.0.3550 (64bit) When i start nxlog it takes up 100%cpu i set the logging mode to debug but i wasnt able to find meaningful information from it. Below is my nxlog configuration where logs are sent to…
sherpaurgen
  • 3,028
  • 6
  • 32
  • 45