0

How can i set S3SinkConnector to stream data from my topic to S3 bucket? I've download plugin from confluent hub, created a file connect-standalone.properties and s3-sink.properties.

When I try to run ./connect-standalone.sh connect-standalone.properties s3-sink.properties, I got an error:

[2019-06-25 15:43:52,841] ERROR Failed to create job for s3-sink.properties (org.apache.kafka.connect.cli.ConnectStandalone:102)
[2019-06-25 15:43:52,842] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:113)
java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches io.confluent.connect.s3.S3SinkConnector

But I can see this plugin is available:

available connectors are: PluginDesc{klass=class io.confluent.connect.s3.S3SinkConnector, name='io.confluent.connect.s3.S3SinkConnector', version='5.2.2', encodedVersion=5.2.2, type=sink, typeName='sink', location='file:/home/ec2-user/kafka-plugins/confluentinc-kafka-connect-s3-5.2.2/'}

Also, I've tried to export CLASSPATH using export CLASSPATH=/home/ec2-user/kafka-plugins and still no lucky.

How can I set S3SinkConnector using AWS MSK?

Alexandre Dupriez
  • 3,026
  • 20
  • 25

1 Answers1

0

You shouldn't be exporting CLASSPATH

You'd have to set up this in the connect-standalone properties 

plugin.path=/home/ec2-user/kafka-plugins
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245