Does anyone know how to change the CloudWatch log output from a "Kinesis Data Analytics for Apache Flink" app.?
There are two things I'd like to change:
- The fields in the JSON written to CloudWatch
- The contents/format of the "message" field (i.e., format of each "LOG.info", "LOG.warn", etc. - line)
#1 is most important.
The default format written to CloudWatch looks like this:
{
"locationInformation": "",
"logger": "",
"message": "",
"threadName": "",
"applicationARN": "arn:aws:kinesisanalytics:eu-west-1:...",
"applicationVersionId": "23",
"messageSchemaVersion": "1",
"messageType": "INFO"
}
Is it somehow possible to change the output, so that each CloudWatch entry becomes this instead:
{
“EventTime”: "20201224T23:59:59.999Z",
“LogLevel”: 5,
“EventSource”: "ApplicationURI/Name",
“Message”: ”foobar”
}
Using SLF4J is mentioned here (https://docs.aws.amazon.com/kinesisanalytics/latest/java/cloudwatch-logs-writing.html), although the format mentioned on the same page is the default described above.
The pom.xml file of the Java project includes aws-java-sdk-logs. It also excludes log4j and slf4j.
<artifactSet>
<excludes>
<exclude>org.slf4j:*</exclude>
<exclude>log4j:*</exclude>
</excludes>
</artifactSet>
I've had a look at this: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-logging.html But when testing locally, changing log4j.properties changes the "message" field of the log entry. That does not seem to be loaded when running on AWS, despite existing in the root directory of the .jar file. Even if I could make aws-java-sdk-logs pick up changes in log4j.properties, changing this file doesn't seem capable of changing the JSON fields written to CloudWatch (only the "message" format).
When the app starts on AWS, I can see it prints this:
-Dlog4j.configuration=file:/etc/flink/log4j-console.properties
-Dlogback.configurationFile=file:/etc/flink/logback-console.xml
I was hoping to perhaps copy them out on startup, changing them, and including them while setting the props to point at the JAR. However, both these files seem empty when trying to read them on startup from inside the Flink app code.
Is there some relatively straight-forward way to:
- Rename/remove/add fields to the JSON written to CloudWatch?
- Change the format of the "message" field?
