We deployed MongoDB on EC2 with r3.8xlarge with 10000 IOPS. We are running a replicaset with 1 primary, 1 secondary, and 1 arbiter. In our applications we have readPreference set to secondaryPreferred.
Now, I am watching the EBS Read Throughput metrics on EBS for Mongo instances. I can see it's hitting 10k IOPs on the Mongo Primary node which is surprising for me; I see spikes on the secondary node which is obvious, but spikes on Primary are what is confusing me?
Why do I see spikes on the Primary mongo node EBS, even though I set readPreference to be secondaryPreferred?