How to create Connector CustomPlugin using AWS_MSK CDK?

Question

I am using CfnConnector (as the below documentation reference link states) to create a MSK connector and it requires a customPluginArn. How am I able to create a custom plugin using the CDK, so i am able to get the CustomPluginArn and the Revision number? There is no any documentation about it. Thanks

For reference class CfnConnector documentation: enter link description here

Custom plugins aren't required. And you'd create plugins externally as JVM projects, then store the ZIP/JAR files in S3, for example, for MSK Connect to download — OneCricketeer, Aug 15 '22 at 14:40
Thanks @OneCricketee. A custom plugin object needs to be created from a jar file stored in S3 in order to be able to create the MSK connector, However, It seems there is not a CDK feature released to allow the creating of the custom plugin object. There is CfnConnector but it only allows the connector creation. — D.B, Aug 19 '22 at 06:18
CloudFormation cannot create plugins. It can _install them_ from S3, as you mention. You can only create Connect plugins from the `connect-api` Kafka Java API — OneCricketeer, Aug 19 '22 at 15:52

score 0 · Answer 1 · answered Aug 23 '22 at 07:06

I ended creating a custom resource, code here if it is useful for someone:

 def _create_custom_plugin(self):
    create_params = {
    "contentType": 'ZIP',
    "location": { 
            's3Location': { 
            'bucketArn': self._msk_connector_bucket_arn,
            'fileKey': self._plugin_file_name, 
            'objectVersion': 'null'
            }
    },            
    "name": self._plugin_name,
    "description": self._plugin_name
    }
     
    create_custom_plugin = custom_resources.AwsSdkCall(
            service='KafkaConnect',
            action='createCustomPlugin',
            region='ap-southeast-2',
            physical_resource_id=custom_resources.PhysicalResourceId.of(f'connector-{self._environment_name}'),
            parameters = create_params
        )

    cr_policy = iam.Policy(self, 'cr_policy',
        statements = [iam.PolicyStatement(
            actions=[
                's3:*'
            ],
            effect=iam.Effect.ALLOW,
            resources=['*']
        )]
    )

    lambda_role = self._get_provisioning_lambda_role(construct_id=id)
    lambda_role.attach_inline_policy(cr_policy)         
            
    create_update_custom_plugin = custom_resources.AwsCustomResource(self,
        'CreateUpdateCustomPlugin',
        on_create=create_custom_plugin,
        on_update=create_custom_plugin,
        policy=custom_resources.AwsCustomResourcePolicy.from_sdk_calls(resources=custom_resources.AwsCustomResourcePolicy.ANY_RESOURCE)
        ,role=lambda_role)
    self._custom_plugin_arn = create_update_custom_plugin.get_response_field('customPluginArn')
    self._revision = create_update_custom_plugin.get_response_field('revision')  


def _get_provisioning_lambda_role(self, construct_id: str):
     return iam.Role(
     scope=self,
     id=f'{construct_id}-LambdaRole',
     assumed_by=iam.ServicePrincipal('lambda.amazonaws.com'),
     managed_policies=[iam.ManagedPolicy.from_aws_managed_policy_name(
        "service-role/AWSLambdaBasicExecutionRole")],
     )

How to create Connector CustomPlugin using AWS_MSK CDK?

1 Answers1