MirrorMaker2 syncs some but not all consumer groups defined by pattern

Question

We are running MirrorMaker2 with a SourceConnector and a CheckpointConnector via Strimzi and try to sync topics from our integration cluster to our dev cluster. Of 43 consumer groups, only 22 are being transfered to the target cluster. We can't find any errors within the logs and the group and topic filters should work. Is there any hint why consumer groups are being ignored or any possibility to identify the cause for this?

Our latest hint was that the debug messages at some point show that the 'non-transferred' groups are 'Dead', even though a describe shortly before showed them as 'Stable'. We guess that this is the reason for MirrorMaker to ignore our consumer groups. See the log messages below. If we check the state of the consumer groups by ourselves (via admin client of kafkactl), they are 'Stable'.

We have the following configuration for MirrorMaker2:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
  name: sync-kafka-from-int-to-dev-85
  namespace: edi-global-dev
spec:
  version: "3.3.1"
  replicas: 1
  connectCluster: "kafka-dev"
  mirrors:
  - sourceCluster: "kafka-int"
    targetCluster: "kafka-dev"
    topicsPattern: ".*"
    groupsPattern: ".*"
    checkpointConnector:
      config:
        checkpoints.topic.replication.factor: 1
        replication.factor: 1 
        sync.group.offsets.enabled: "true"
        replication.policy.class: "io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy"
        refresh.groups.interval.seconds: 30
        refresh.topics.interval.seconds: 30
    sourceConnector:
      config:
        replication.factor: 1 
        offset-syncs.topic.replication.factor: 1
        offset-syncs.topic.location: target
        sync.group.offsets.enabled: "true"
        sync.topic.acls.enabled: "false" 
        replication.policy.class: "io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy"
  clusters:
  - alias: "kafka-int"
    authentication:
      ...
    bootstrapServers: kafka-int.svc.cluster.local:9093
    tls:
      trustedCertificates:
      - certificate: ca.crt
        secretName: "kafka-int-cluster-ca-cert"
  - alias: "kafka-dev"
    authentication:
      ...
    bootstrapServers: kafka-dev.svc.cluster.local:9093
    config:
      config.storage.replication.factor: 1
      offset.storage.replication.factor: 1
      status.storage.replication.factor: 1
    tls:
      ...

Exemplary log where Groupd is 'Stable' and then seen as 'Dead'.

2023-04-27 14:10:46,197 DEBUG [kafka-int->kafka-dev.MirrorCheckpointConnector|worker]
 [AdminClient clientId=adminclient-11] Received LIST_GROUPS response from node 2 for 
request with header RequestHeader(apiKey=LIST_GROUPS, apiVersion=4, clientId=adminclient-
11, correlationId=4): ListGroupsResponseData(throttleTimeMs=0, errorCode=0, groups=[..., 
ListedGroup(groupId='xxx-my-consumer-group', protocolType='consumer', groupState='Stable'), ...])


2023-04-27 14:10:46,211 DEBUG [kafka-int->kafka-dev.MirrorCheckpointConnector|task-0] [AdminClient clientId=adminclient-17] 
Received DESCRIBE_GROUPS response from node 0 for request with header RequestHeader(apiKey=DESCRIBE_GROUPS, apiVersion=5, clientId=adminclient-17, 
correlationId=4): DescribeGroupsResponseData(throttleTimeMs=0, groups=[..., 
DescribedGroup(errorCode=0, groupId='xxx-my-consumer-group', groupState='Dead', 
protocolType='', protocolData='', members=[], authorizedOperations=-2147483648), ...

MirrorMaker2 syncs some but not all consumer groups defined by pattern

0 Answers0