Application logging with object serialization

Question

I'm implementing logging in web service application with following requirements:

• log should be stored in database
• log should be machine-readable (every bit of information should be stored in separate column)
• log should be extensible (client code can specify info which will go particular column in database)
• should be able to pass big object from client code to database (serialiazing)
• should not hit performance (DB writing operations should be done in separate thread)

I know that log4net and similair solutions have DB appenders. But what about deffered DB writing? And big objects?

The basic use case for this is ability to look through the events and be able to get input/output objects at any point of execution.

I feel like I messing application logging with something different. Does anyone know correct name for such product/architecture? Maybe there are some generic solutions?

Answer 1

Give ReflectInsight a try. It uses a structure format with the ability to add extended properties. It also has a Db writer listener or you can create your own.

Edited:

1. log should be stored in database (Yes, you can use their DB Listener for this)
2. log should be machine-readable (every bit of information should be stored in separate column) (Yes, there are standard properties that are store for every message, plus you can define your own by using extended properties, which are also stored to the DB and other listeners, like the live Viewer, etc.)
3. log should be extensible (client code can specify info which will go particular column in database) should be able to pass big object from client code to database (serialiazing) (Yes, this is out of the box. You can configure what properties you want to store in the DB by simple configuration. Custom objects are automatically serialized in the logs (or DB in your case), as RI uses a structure format for logging)
4. should not hit performance (DB writing operations should be done in separate thread) (Yes. All logging activities are sent to listeners via a separate worker thread in order not to affect the host application's performance)
5. The Live Viewer alone can receive 80,000 msg/sec
6. Live memory footprint is small as most msgs are cached on hard drive
7. Has Auto Save/Auto Purge capabilities.
8. Can easily use NLog, Log4net, EntLib, Common Logging frameworks to map to RI's Framework (However, you will lose the ability to log rich details such as Datasets, Collections, etc).

DISCLAIMER: I don't directly work for ReflectSoftware but I'm one of the main developers that help build ReflectInsight. My main goal is help anyone with their logging framework needs and only respond to stackoverflow questions that are applicable in such regard.