Specify credentials to AWS Cloudwatch logs Client

Question

Hi I am trying to create a exception logs of my java application code in AWS cloudwatch for that I have used CloudWatchLogsClient to put my events to it but i am getting a below Error

DEBUG software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain - Unable to load credentials from SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId).
software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId).
    at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:97)
    at software.amazon.awssdk.auth.credentials.internal.SystemSettingsCredentialsProvider.resolveCredentials(SystemSettingsCredentialsProvider.java:58)
    at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:91)
    at software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:52)
    at software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:100)
    at software.amazon.awssdk.awscore.client.handler.AwsClientHandlerUtils.createExecutionContext(AwsClientHandlerUtils.java:71)
    at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.createExecutionContext(AwsSyncClientHandler.java:68)
    at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:68)
    at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:44)
    at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
    at software.amazon.awssdk.services.cloudwatchlogs.DefaultCloudWatchLogsClient.describeLogStreams(DefaultCloudWatchLogsClient.java:1168)
    at com.WorkingwithS3.WorkingwithS3.PutLogEvents.main(PutLogEvents.java:58)

Here is my code sample

package com.WorkingwithS3.WorkingwithS3;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsClient;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsClientBuilder;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeLogStreamsRequest;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeLogStreamsResponse;
import software.amazon.awssdk.services.cloudwatchlogs.model.InputLogEvent;
import software.amazon.awssdk.services.cloudwatchlogs.model.PutLogEventsRequest;

import java.util.Arrays;
// snippet-end:[cloudwatch.java2.put_log_events.import]

/**
 * Puts a sample CloudWatch log event
 */
public class PutLogEvents {
    public static void main(String[] args) {
        BasicAWSCredentials creds = new BasicAWSCredentials("xxxx",
                "xxxxx");

//        BasicAWSCredentials creds = new BasicAWSCredentials("xxxxxxxx",
//                "xxxx");

        String regionId = "xxx";
        String logGroupName = "xxxx";
        String streamName = "xxxxx";


        // snippet-start:[cloudwatch.java2.put_log_events.main]

        CloudWatchLogsClient logsClient = CloudWatchLogsClient.builder().region(Region.of(regionId)).build();

        // A sequence token is required to put a log event in an existing stream.
        // Look up the stream to find its sequence token.

        // First describe all streams in the log group.
        DescribeLogStreamsRequest logStreamRequest = DescribeLogStreamsRequest.builder()
                .logGroupName(logGroupName)
                .logStreamNamePrefix(streamName)
                .build();
        DescribeLogStreamsResponse describeLogStreamsResponse = logsClient.describeLogStreams(logStreamRequest);

        // Assume that a single stream is returned since a specific stream name was specified in the previous request.
        String sequenceToken = describeLogStreamsResponse.logStreams().get(0).uploadSequenceToken();

        // Build an input log message to put to CloudWatch.
        InputLogEvent inputLogEvent = InputLogEvent.builder()
                .message("{ \"key1\": \"value1\", \"key2\": \"value2\" }")
                .timestamp(System.currentTimeMillis())
                .build();

        // Specify the request parameters.
        PutLogEventsRequest putLogEventsRequest = PutLogEventsRequest.builder()
                .logEvents(Arrays.asList(inputLogEvent))
                .logGroupName(logGroupName)
                .logStreamName(streamName)
                // Sequence token is required so that the log can be written to the
                // latest location in the stream.
                .sequenceToken(sequenceToken)
                .build();
        logsClient.putLogEvents(putLogEventsRequest);
        // snippet-end:[cloudwatch.java2.put_log_events.main]

        System.out.println("Successfully put CloudWatch log event");
    }
}

Could anyone please guide how to specify the credentials for CloudWatchLogsClient? Thanks in advance

from your trace your best option would be to set your credentials() as environment variables see https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html — Nigel Savage, Jan 06 '20 at 14:13
Is there any possible way to set the credentials with in my java code itself? — Mohan vel, Jan 07 '20 at 12:00

Nigel Savage · Accepted Answer · 2020-01-13T19:14:15.317

1

From the trace we can see the sdk client in this instance CloudWatchLogsClient.builder() is failing to find the credentials and hence failing to build.
The client will look for the credentials in the following defaults locations

For a lot of reasons its good to set up your code to read the credentials from environment variables.

This follows for many reasons.

AWS encourage the use of environment variables for credentials.

The increasing need to run your application in some kind of a container cluster like Kubernetes for example.

Often in a containerized environment access to the file system can be problematic.

In many container tools like docker-compose its trivial to pass environment variables to the container.

In the link defaults locations it specifies the options precisely how to supply the credentials for the CloudWatchLogsClient.builder() operation and for the reasons above suggest you adopt the environment variables solution and you can test that they are set correctly by using `

       Map<String, String> mapOfEnvironmentVariables = System.getenv();

to retrieve them.

edited Jan 13 '20 at 19:14

answered Jan 07 '20 at 14:32

Nigel Savage

991
13
26

In my case we are going to deploy all the Java services in docker container in that case which should be the best approach? – Mohan vel Jan 07 '20 at 14:49
If i'am right i have to pass the credentials as Environment variables? – Mohan vel Jan 07 '20 at 14:52
1

when the CloudWatchLogsClient.builder() runs and the creds are in the environment which you can test for with System.getenv() then the client should authenticate. Just how you supply the environment variable(s) depends on the system you will deploy your container too. – Nigel Savage Jan 07 '20 at 14:53
1

@Mohan vel something went wrong with the last edit u made to my answer. . – Nigel Savage Jan 13 '20 at 19:09

score 1 · Answer 2 · answered Jan 20 '20 at 14:17

Below code working fine i am able to write the exception in cloudwatch using CloudWatchLogsClient just for reference i have attached code

package com.example.DynamoDB;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsClient;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeLogStreamsRequest;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeLogStreamsResponse;
import software.amazon.awssdk.services.cloudwatchlogs.model.InputLogEvent;
import software.amazon.awssdk.services.cloudwatchlogs.model.PutLogEventsRequest;

import java.util.Arrays;

@ControllerAdvice
public class ExceptionControllerAdvice {
    @ExceptionHandler(Exception.class)
    public ResponseEntity<ErrorResponse> exceptionHandler(Exception ex) {
        ErrorResponse error = new ErrorResponse();
        error.setErrorCode(HttpStatus.INTERNAL_SERVER_ERROR.value());
        error.setMessage(ex.getMessage());
        error.setController(ex.getStackTrace()[0].getClassName());
        error.setService(ex.getStackTrace()[0].getClassName());
        error.setTimestamp(System.currentTimeMillis());
        PutLogEvents(error);
        return new ResponseEntity<ErrorResponse>(error, HttpStatus.OK);
    }

    public static void PutLogEvents(ErrorResponse Er)
    {
        String regionId = "us-east-1";
        String logGroupName = "xxxxxxxxx";
        String logStreamName = "xxxxxxxxx";

        CloudWatchLogsClient logsClient = CloudWatchLogsClient.builder().region(Region.of(regionId)).build();

        // A sequence token is required to put a log event in an existing stream.
        // Look up the stream to find its sequence token.
        String sequenceToken = getNextSequenceToken(logsClient, logGroupName, logStreamName);

        // Build a JSON log using the EmbeddedMetricFormat.

        String message = "[{" +
                "  \"Timestamp\": " + Er.getTimestamp()  + "," +
                "  \"ErrorCode\": " + Er.getErrorCode()  + "," +
                "  \"ControllerName\": " + Er.getErrorCode()  + "," +
                "  \"ServiceName\": " + Er.getErrorCode()  + "," +
                "  \"ErrorMsg\": " + Er.getErrorCode()   + "" +
                "}]";
        InputLogEvent inputLogEvent = InputLogEvent.builder()
                .message(message)
                .timestamp(Er.getTimestamp())
                .build();

        // Specify the request parameters.
        PutLogEventsRequest putLogEventsRequest = PutLogEventsRequest.builder()
                .logEvents(Arrays.asList(inputLogEvent))
                .logGroupName(logGroupName)
                .logStreamName(logStreamName)
                // Sequence token is required so that the log can be written to the
                // latest location in the stream.
                .sequenceToken(sequenceToken)
                .build();

        logsClient.putLogEvents(putLogEventsRequest);
    }

    private static String getNextSequenceToken(CloudWatchLogsClient logsClient, String logGroupName, String logStreamName) {
        DescribeLogStreamsRequest logStreamRequest = DescribeLogStreamsRequest.builder()
                .logGroupName(logGroupName)
                .logStreamNamePrefix(logStreamName)
                .build();

        DescribeLogStreamsResponse describeLogStreamsResponse = logsClient.describeLogStreams(logStreamRequest);

        // Assume that a single stream is returned since a specific stream name was
        // specified in the previous request.
        return describeLogStreamsResponse.logStreams().get(0).uploadSequenceToken();
    }

}

Specify credentials to AWS Cloudwatch logs Client

2 Answers2