0

As the question states, when the MSK was set up - it had no security enabled, and in turn it's causing issues creating the rule itself.

When creating the rule through the AWS GUI, the action username and password property must be configured using the "get_secret function" - I initially configured our CloudFormation like so:

  IoTCoreToKafkaRule:
    Type: AWS::IoT::TopicRule
    Properties: 
      RuleName: !Sub "IoTCoreToKafkaRule"
      TopicRulePayload: 
        Description: Rule to forward MQTT messages to MSK
        RuleDisabled: false
        AwsIotSqlVersion: "2016-03-23"
        Sql: "SELECT encode(*,'base64') AS message, topic() AS topic FROM 'topic/#'"
        Actions:
          - Kafka:
              DestinationArn: !GetAtt TopicRuleVpcDestination.Arn
              Topic: "iotcore"
              ClientProperties:
                  'sasl.mechanism': SCRAM-SHA-512
                  'security.protocol': SASL_SSL
                  'bootstrap.servers': !Ref BootstrapServers
                  'sasl.scram.password': "DummyUsername"
                  'sasl.scram.username': "DummyPassword"

I was thinking that I can supply "Dummy" values and since the MSK has no security it would technically ignore this config and let the data through. Now, I understand there is a case to say "what if you remove everything aside from the bootstrap.servers" - well this causes an issue and I receive an error that the ClientProperties need either SASL_SSL or SSL configuration, just like in the AWS GUI.

Is there a way for me to create the rule, without the need for enabling SASL_SSL or SSL security on the MSK? Do I need to create a Secret with any odd values that it can pull from or is there an easier solution?

veila
  • 83
  • 8

1 Answers1

0

Okay, seems like I answer my own issues these days.

Well it turns out I was on the right lines, although AWS doesn't have the option to create a rule to the Kafka if there is no security, you can create the values with any username/password as long as you retrieve it from a secrets manager.

With the use of this repository and my the code above, plus the addititon of getting the password in the form of ${get_secret("")), I was able to satisfy the values for security, but as far as I know aside from the usual, VPC/Subnet/Bootstrap brokers, I didn't need to satisfy any other value.

veila
  • 83
  • 8