Camel S3: Listing S3 bucket files with scheduler turned off

Question

I am having following Camel route which is trying to read the list of files from an S3 bucket:

from("direct:my-route").
.from("aws-s3://my.bucket?useIAMCredentials=true&useAwsKMS=true&awsKMSKeyId=my-key-id&deleteAfterRead=false&operation=listObjects&includeBody=false&prefix=test1/test.xml")
.log(" File detected: ${header.CamelAwsS3Key}")
.end();

However this route is being called by an external scheduler which is running every minute. It looks like the default behaviour of the Camel-S3 component is to run with a scheduler however this is causing the same files being processed again and again.

I have tried to turn the Camel-S3 scheduler off with startScheduler=false however this does not execute the 'aws-s3' part when the external scheduler kicks in and getting null values for '${header.CamelAwsS3Key}'.

Is it possible to run this component without the internal scheduler?

Camel version being used - 2.22.0

Dependency used for aws:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-aws</artifactId>
  <version>${camel.version}</version>
</dependency>

score 2 · Answer 1 · answered Jan 10 '20 at 04:31

Dont have 2 x from, that is not basically two independent consumers. Instead use a content enricher (pollEnrich) to consume from s3 when the other from is called.

from
   pollEnrich
   log

Read the docs about content enricher and pollEnrich / enrich (specially around timeouts with poll enrich). https://camel.apache.org/manual/latest/content-enricher.html

Camel S3: Listing S3 bucket files with scheduler turned off

1 Answers1