I'm working on k8s custom resource upgrade.
new resource: { spec: {}, status: { extraField: x }}
old resource: { spec: {}, status: {}}
new resource was deployed in the k8s cluster but due to some reason I had to rollback to old resource. The kubernetes watch is being used internally in the program to monitor resource changes. But now the watch receives the new resource time to time, initially I thought its because of the event cache or something with the cluster and it stop receiving it. but after days the new resource began to receive in the watch, then after few hours or so it went out and rollback resource (old resource) began to receive as it should.
timeline
1. new resource deployed -> available and checked in k8s cluster
2. watch receives new resource
3. due to some reason new resource rollback -> old resource is available in k8s and checked
4. watch receives old resource
5. watch receives new resource for arbitrary time period and stops
6. receiving new resource is stopped
7. smooth run for few days
8. watch receives new resource for few hours and stops
9. smooth run
Why time to time it receives a custom resource data that is not exists anymore?
I have tried with checking the caches, querying the custom resource using kubectl and found nothing about extraField that kubernetes client WatchConnectionManager receives time to time. Any idea or clue is highly appreciated.
