1

I am writting custom logs to log analytics.

Based on the following link:

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#create-a-request

I should be able to use a field in the request header called: time-generated-field, the documentation says that: " If you specify a field, its contents are used for TimeGenerated. If you don't specify this field, the default for TimeGenerated is the time that the message is ingested. The contents of the message field should follow the ISO 8601 format YYYY-MM-DDThh:mm:ssZ"

I am passing the following value: 2021-11-11T19:52:45Z (as a string, since you can't pass this as a datetime object) but the problem is that when I look in the log analytics workspace, the TimeGenerated field is this (today's date): 2021-12-01T18:41:04.529Z which is the datetime the event is ingested, so basically, it's not taking the real event generated time which is 2021-11-11T19:52:45Z which I am passing in the header.

Am I doing something wrong here?

Any help would be appreciated, I am running out of ideas here.

Rakim
  • 167
  • 11
  • 1
    Glad that your issue was fixed. You can accept and upvote the answer so that it will be helpful to other community members who might face the same issue. – Ecstasy Dec 17 '21 at 04:30

1 Answers1

2

It turns out I misread the documentation and didn't realize that the time-generated-field is not an argument you pass in, but a property in the body of the JSON data that you are sending to the log analytics workspace.

It can be referenced in this manner (this is an example of 1 way of doing it):

def post_data(customer_id, shared_key, body, log_type):
    method = 'POST'
    #The string "raised" assigned to the TimeStampField variable below 
    #is an actual property in the JSON object that I am sending to log 
    #analytics, it contains the datetime in the expected format.
    TimeStampField = "raised" 
    content_type = 'application/json'
    resource = '/api/logs'
    rfc1123date = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
    content_length = len(body)
    signature = build_signature(customer_id, shared_key, rfc1123date, content_length, method, content_type, resource)
    uri = 'https://' + customer_id + '.ods.opinsights.azure.com' + resource + '?api-version=2016-04-01'
    headers = {
        'content-type': content_type,
        'Authorization': signature,
        'Log-Type': log_type,
        'x-ms-date': rfc1123date,
        'time-generated-field': TimeStampField
    }

Will definitely create a pull request to modify the wording here: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#create-a-request As I feel it is not very intuitive

Rakim
  • 167
  • 11
  • Also a tip for someone who comes to this problem. `x-ms-date` header and date you put into authorization signature should be `DateTime.UtcNow` and the field specified in `time-generated-field` header is the actual log time. – donatasj87 Feb 20 '23 at 13:06