Metrics not exporting for cp-ksql-server and cp-zookeeper for confluent kafka installation from cp-helm-charts

Question

I'm trying to setup monitoring for confluent kafka installed from latest cp-helm-charts. The issue is that metrics are exported fine for cp-kafka but not cp-ksql-server and cp-zookeeper.

1. JMX prom exporter side car is configured fine.
2. ServiceMonitors for cp-kafka, cp-ksql-server and cp-zookeeper are configured fine.

Able to see the metrics status up for cp-kafka, cp-ksql-server and cp-zookeeper on https://my-prometheus:9090/targets

The catch here is, when I view

http://my-cp-kafka:5556/metrics

kafka metrics show up fine. Like so...

# HELP cp_kafka_controller_kafkacontroller_activecontrollercount Attribute exposed for management (kafka.controller<type=KafkaController, name=ActiveControllerCount><>Value)
# TYPE cp_kafka_controller_kafkacontroller_activecontrollercount untyped
cp_kafka_controller_kafkacontroller_activecontrollercount 1.0
# HELP cp_kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems Attribute exposed for management (kafka.controller<type=ControllerStats, name=TopicUncleanLeaderElectionEnableRateAndTimeMs><>OneMinuteRate)
# TYPE cp_kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems untyped
cp_kafka_controller_controllerstats_topicuncleanleaderelectionenablerateandtimems 0.0
# HELP cp_kafka_controller_controllerstats_updatefeaturesrateandtimems Attribute exposed for management (kafka.controller<type=ControllerStats, name=UpdateFeaturesRateAndTimeMs><>OneMinuteRate)
# TYPE cp_kafka_controller_controllerstats_updatefeaturesrateandtimems untyped
cp_kafka_controller_controllerstats_updatefeaturesrateandtimems 0.0
# HELP cp_kafka_controller_controllerstats_partitionreassignmentrateandtimems Attribute exposed for management (kafka.controller<type=ControllerStats, name=PartitionReassignmentRateAndTimeMs><>OneMinuteRate)
# TYPE cp_kafka_controller_controllerstats_partitionreassignmentrateandtimems untyped
cp_kafka_controller_controllerstats_partitionreassignmentrateandtimems 0.0
# HELP cp_kafka_server_brokertopicmetrics_failedfetchrequestspersec Attribute exposed for management (kafka.server<type=BrokerTopicMetrics, name=FailedFetchRequestsPerSec><>OneMinuteRate)
# TYPE cp_kafka_server_brokertopicmetrics_failedfetchrequestspersec untyped
cp_kafka_server_brokertopicmetrics_failedfetchrequestspersec 0.0
# HELP cp_kafka_controller_controllerstats_logdirchangerateandtimems Attribute exposed for management (kafka.controller<type=ControllerStats, name=LogDirChangeRateAndTimeMs><>OneMinuteRate)
# TYPE cp_kafka_controller_controllerstats_logdirchangerateandtimems untyped
cp_kafka_controller_controllerstats_logdirchangerateandtimems 0.0
# HELP cp_kafka_server_brokertopicmetrics_invalidmagicnumberrecordspersec Attribute exposed for management (kafka.server<type=BrokerTopicMetrics, name=InvalidMagicNumberRecordsPerSec><>OneMinuteRate)
# TYPE cp_kafka_server_brokertopicmetrics_invalidmagicnumberrecordspersec untyped
cp_kafka_server_brokertopicmetrics_invalidmagicnumberrecordspersec 0.0
...
...
# TYPE cp_kafka_server_sessionexpirelistener_zookeeperexpirespersec untyped
cp_kafka_server_sessionexpirelistener_zookeeperexpirespersec 2.964393875E-314
# HELP jmx_scrape_duration_seconds Time this JMX scrape took, in seconds.
# TYPE jmx_scrape_duration_seconds gauge
jmx_scrape_duration_seconds 0.053273124
# HELP jmx_scrape_error Non-zero if this scrape failed.
# TYPE jmx_scrape_error gauge
jmx_scrape_error 0.0
# HELP jmx_config_reload_success_total Number of times configuration have successfully been reloaded.
# TYPE jmx_config_reload_success_total counter
jmx_config_reload_success_total 0.0
# HELP jmx_config_reload_failure_total Number of times configuration have failed to be reloaded.
# TYPE jmx_config_reload_failure_total counter
jmx_config_reload_failure_total 0.0

BUT when I view

http://my-cp-ksql-server:5556/metrics

and

http://my-cp-zookeeper:5556/metrics

the metrics are shown empty. Only text I see is like this without any metrics.

# HELP jmx_scrape_duration_seconds Time this JMX scrape took, in seconds.
# TYPE jmx_scrape_duration_seconds gauge
jmx_scrape_duration_seconds 0.003530826
# HELP jmx_scrape_error Non-zero if this scrape failed.
# TYPE jmx_scrape_error gauge
jmx_scrape_error 0.0
# HELP jmx_config_reload_success_total Number of times configuration have successfully been reloaded.
# TYPE jmx_config_reload_success_total counter
jmx_config_reload_success_total 0.0
# HELP jmx_config_reload_failure_total Number of times configuration have failed to be reloaded.
# TYPE jmx_config_reload_failure_total counter
jmx_config_reload_failure_total 0.0