ClusterPodMonitring defines how to scrape all pod monitors in cluster that have some label. In what namespace we sholud define this CR in kubernetes cluster. What is best pratice?
Somewhere I read it is used "This resource is a specialized tool intended for cluster-scoped metrics (such as kube-state-metrics)." Why in this case we cannot use PodMonitoring?
https://github.com/GoogleCloudPlatform/prometheus-engine/blob/v0.3.1/doc/
PodMonitoring is basically identical to ClusterPodMonitoring.
They only differ by spec.scope in the CustomResourceDefinition like you can see in their GitHub repo. The latter is a cluster-wide definition, while PodMonitoring is namespaced. Depending on your needs you can freely choose which one to use - either you want to separate your scrape-jobs per namespace, or you don't.
A ClusterPodMonitoring resource can discover targets across all namespaces in a cluster, so best practice would be to spawn it in it's own namespace, e.g. named monitoring.
A PodMonitoring resource can only discover targets in it's own namespace.
Further on it's up to you which labels you want to be matched by (Cluster)PodMonitoring.spec.selector. It's always a good idea to be consistent all over you clusters, independent if you use cluster-wide or namespaced monitoring.
tldr; A CRD in kubernetes can be defined in the scope Namespaced or Cluster (k8s docs), defining which resources they can access. Google did this with ClusterPodMonitoring.
