0

I've enabled diagnostics logging to Blob Storage for an Azure Website I am trying on. I've also set Nlog to write to Trace, so that they are then in turn written to the Azure blob. Nlog layout is set to CSV. This works, and the generated logs are outputted to the blob storage. If this was logging to a traditional file, that file would be a CSV file which I can open in excel, to analyse better the log files.

Nlog configuration file copied below:

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <!-- 
  See http://nlog-project.org/wiki/Configuration_file 
  for information on customizing logging rules and outputs.
   -->
  <targets>
    <!-- add your targets here -->

    <!--
    <target xsi:type="File" name="f" fileName="${basedir}/logs/${shortdate}.log"
            layout="${longdate} ${uppercase:${level}} ${message}" />
    -->
    <target xsi:type="Trace" name="trace" >
      <layout xsi:type="CsvLayout" >
        <column name="shortdate" layout="${shortdate}" />
        <column name="time" layout="${time}" />
        <column name="logger" layout="${logger}"/>
        <column name="level" layout="${level}"/>
        <column name="machinename" layout="${machinename}"/>
        <column name="processid" layout="${processid}"/>
        <column name="threadid" layout="${threadid}"/>
        <column name="threadname" layout="${threadname}"/>
        <column name="message" layout="${message}" />
        <column name="exception" layout="${exception:format=tostring}" />

      </layout>
    </target>
  </targets>

  <rules>
    <!-- add your logging rules here -->
    <logger name="*" minlevel="Trace" writeTo="trace" />
    <!--
    <logger name="*" minlevel="Trace" writeTo="f" />
    -->
  </rules>
</nlog>

Windows Azure diagnostics saves the diagnostics info as a CSV file in the blob storage. The CSV file has the below columns.

date,level,applicationName,instanceId,eventTickCount,eventId,pid,tid,message,activityId

However, the entire NLog message is written in the Message column. This is probably because it saves the Diagnostics.Trace message there, in which NLog is saving it'slogs. For example:

2014-05-07T12:18:49,Information,KarlCassarTestAzure1,10cd67,635350619297036217,0,2984,1,"2014-05-07,12:18:49.6254,TestAzureWebApplication1.MvcApplication,Info,RD0003FF410F59,2984,1,,Application_Start,",

The NLog message is the below:

"2014-05-07,12:18:49.6254,TestAzureWebApplication1.MvcApplication,Info,RD0003FF410F59,2984,1,,Application_Start,"

It is escaped, and fits entirely in the CSV column, which I wouldn't wish. Any idea if there is something to do about this?

Karl Cassar
  • 6,043
  • 10
  • 47
  • 84

1 Answers1

0

For what you're trying to accomplish, you would need to rely on Custom Diagnostics functionality of Azure Diagnostics. The way it would work is that your NLog logger would write log files in CSV format to local storage in your cloud service VM and then Azure Diagnostics module will transfer this data to blob storage periodically.

For implementation details, I would recommend looking into Cloud Service Fundamentals project by Azure CAT team. They have described the process quite extensively in their blog post which you can read here: http://social.technet.microsoft.com/wiki/contents/articles/17987.cloud-service-fundamentals.aspx. What you would want to do there is read all blog posts under Telemetry section. For the services we have built, we have actually implemented this and it works just great.

Gaurav Mantri
  • 128,066
  • 12
  • 206
  • 241
  • @GuaravMantri I'll go through the blog post. However, does it merge log files from different instances into one blob file? – Karl Cassar May 07 '14 at 13:21
  • No it doesn't. Each instance writes its own log file in blob storage. What you could do is have a background process combine it or may be parse it and push it in table storage. – Gaurav Mantri May 07 '14 at 13:49
  • I'm not sure neither if the "normal" Azure Diagnostics actually merges them or not, something I'll need to reconfirm once I have multiple instances. Then again, merging them would be a minor issue. – Karl Cassar May 07 '14 at 13:52
  • Azure diagnostics won't merge them ... you have to spin off a worker role to do that (i.e. you would need to write code for merging). – Gaurav Mantri May 07 '14 at 13:53