Spring Cloud Function Was S3Event error converting generic message

Question

I'm using the last version of Spring Cloud Function as below

<properties>
    <java.version>8</java.version>
    <spring-cloud.version>2020.0.1</spring-cloud.version>
    <aws-lambda-java-core.version>1.2.1</aws-lambda-java-core.version>
    <aws-lambda-java-events.version>3.7.0</aws-lambda-java-events.version>
    <aws-java-sdk-s3.version>1.11.948</aws-java-sdk-s3.version>
    <wrapper.version>1.0.17.RELEASE</wrapper.version>
</properties>
<dependencies>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-function-context</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-function-adapter-aws</artifactId>
    </dependency>

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-s3</artifactId>
        <version>${aws-java-sdk-s3.version}</version>
    </dependency>

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-lambda-java-core</artifactId>
        <version>${aws-lambda-java-core.version}</version>
    </dependency>

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-lambda-java-events</artifactId>
        <version>${aws-lambda-java-events.version}</version>
    </dependency>

this is the main class

public static void main(String[] args) {
    System.out.println("Application started");
    SpringApplication.run(Application.class, args);
}


@Bean
public Function<S3Event, String> extractFileName() {
    return s3Event -> {
        return s3Event.getRecords().get(0).getS3().getBucket().getName();
    };
}

I'm running the application using AWS SAM

sam local invoke AwsLambdaS3Local --log-file ./output.log -e event.json

The event.json content is

{"Records":[{"eventVersion":"2.0","eventTime":"1970-01-01T00:00:00.000Z","requestParameters":{"sourceIPAddress":"127.0.0.1"},"s3":{"configurationId":"testConfigRule","object":{"eTag":"0123456789abcdef0123456789abcdef","sequencer":"0A1B2C3D4E5F678901","key":"test.txt","size":1024},"bucket":{"arn":"arn:aws:s3:::demo_bucket","name":"demo_bucket","ownerIdentity":{"principalId":"EXAMPLE"}},"s3SchemaVersion":"1.0"},"responseElements":{"x-amz-id-2":"EXAMPLE123\/5678abcdefghijklambdaisawesome\/mnopqrstuvwxyzABCDEFGH","x-amz-request-id":"EXAMPLE123456789"},"awsRegion":"us-east-1","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"EXAMPLE"},"eventSource":"aws:s3"}]}

When the application starts, I see the below error

Caused by: java.lang.IllegalArgumentException: Failed to convert input: GenericMessage [payload=byte[676], headers={date=Fri, 05 Feb 2021 12:59:47 GMT, content-length=676, lambda-runtime-aws-request-id=3a84db79-8fc8-44e3-b223-63d20008cf8c, id=5bde8b80-a881-4c4d-222b-8d0a46c0cb55, contentType=application/json, lambda-runtime-deadline-ms=3225038924621, timestamp=1612529988064}] to class com.amazonaws.services.lambda.runtime.events.S3Event

Any idea? Do you have a sample of Spring Cloud Function 2020.0.1 working?

Thank you

This looks like a bug. Can you please raise an issue in - https://github.com/spring-cloud/spring-cloud-function/issues? — Oleg Zhurakousky, Feb 05 '21 at 14:23
Also, would be nice to see a stack trace leading to `Caused by: java.lang.IllegalArgumentException: Failed to convert input:`. In any event, there is a [test case](https://github.com/spring-cloud/spring-cloud-function/blob/c4dfffe0bac5b0d6924d0a868d5757c3c32debee/spring-cloud-function-adapters/spring-cloud-function-adapter-aws/src/test/java/org/springframework/cloud/function/adapter/aws/FunctionInvokerTests.java#L541) and i just tested with your input and everything worked, so need a bit more info. — Oleg Zhurakousky, Feb 05 '21 at 15:25
What I also find interesting is that the length of the input you provided is 1106 - `GenericMessage [payload=byte[1106],. . .`, but in your exception is's 676 — Oleg Zhurakousky, Feb 05 '21 at 15:28
That's even more puzzling given my answer below, since nothing really changed related to AWS versions between 2020.0.0 and 2020.0.1 — Oleg Zhurakousky, Feb 05 '21 at 15:54

score 1 · Accepted Answer · answered Feb 05 '21 at 15:52

Digging some more, you have version missmatch. Basically at the moment we're dependent on the following dependencies from AWS

<aws-lambda-events.version>2.2.6</aws-lambda-events.version>
<aws-java-sdk.version>1.11.825</aws-java-sdk.version>

you are using

<aws-lambda-java-events.version>3.7.0</aws-lambda-java-events.version>
<aws-java-sdk-s3.version>1.11.948</aws-java-sdk-s3.version>

Not sure about all the details but I see that the structure of S3EEvent has changed. In other words there is a breaking change in AWS S3Event and your actual event.json is not compatible with it. You can easily validate it with the following code

try {
    ObjectMapper mapper = new ObjectMapper();
    mapper.readValue(<get bytes from your event.json>, S3Event.class);
}
catch (Exception e) {
    e.printStackTrace();
}

Tryed with the lower version you shared, same event.json, it worked! — Premier, Feb 05 '21 at 16:20

Spring Cloud Function Was S3Event error converting generic message

1 Answers1