0

I am trying to send the files from SFTP server to S3 Bucket. I wrote a spring-boot application using apache camel and it works perfectly fine for one node. I want to scale my application to multiple nodes. To avoid processing same file by multiple nodes, i am using HazelcastIdempotentRepository using this tutorial. 

from(source)
    .pollEnrich(source)
    .log("Uploading file ${file:name} started...")
    .setHeader(S3Constants.CONTENT_LENGTH, simple("${in.header.CamelFileLength}"))
    .setHeader(S3Constants.KEY, simple("${in.header.CamelFileNameOnly}"))
    .toF("hazelcast:%sfoo", HazelcastConstants.SEDA_PREFIX);

fromF("hazelcast:%sfoo", HazelcastConstants.SEDA_PREFIX)
     .idempotentConsumer(simple("*"), hazelcastIdempotentRepo)
     .setHeader(HazelcastConstants.OBJECT_ID, simple("${exchangeId}"))
     .setHeader(HazelcastConstants.OPERATION, constant(HazelcastConstants.PUT_OPERATION))
     //.setHeader(S3Constants.CONTENT_LENGTH, simple("${in.header.CamelAwsS3ContentLength}"))
     .setHeader(S3Constants.KEY, header(S3Constants.KEY))  // How to get file name here ??
     .toF(destination)
     .log("Uploading file ${file:name} completed...");

I am getting the following exception

java.lang.IllegalArgumentException: AWS S3 Key header missing.

This works when i am polling on SFTP Endpoint. 

.setHeader(S3Constants.KEY, simple("${in.header.CamelFileNameOnly}"))

If i hardcode the filename as below, it works as expected. 

.setHeader(S3Constants.KEY, constant("test.txt"))

How do i get fileName header from Hazelcast Queue?

Kenster
  • 23,465
  • 21
  • 80
  • 106
Gangaraju
  • 4,406
  • 9
  • 45
  • 77

1 Answers1

0

Refer to Camel documentation for File Language at

http://camel.apache.org/file-language.html

Use simple language (http://camel.apache.org/simple.html) to grab the only the name part with extn using ${file:onlyname}

Since you are transferring from SFTP to S3, you might need the entire path as the key, where ${file:name} would be helpful

Siva
  • 167
  • 2
  • 9