Hide Forgot
Description of problem: In IPI mode, cluster-image-registry-operator calls ListKeys on the Azure storage account incessantly. This is bad behaviour, especially so because the quotas on ListKeys() calls are low (https://aka.ms/srpthrottlinglimits) and others could be affected. Ideally, cluster-image-registry-operator should call ListKeys once at start-up, and then subsequently in a rate-limited way if it detects an authentication error using the key that it has cached (e.g. in the unlikely event that the end user has rotated the storage key). Version-Release number of selected component (if applicable): 4.3.27 How reproducible: Always. Steps to Reproduce: 1. Run IPI cluster. 2. Check audit logs on registry storage account in Azure portal. Actual results: Lots of ListKeys calls seen. Expected results: Few ListKeys calls seen. Additional info: This BZ is closely associated to https://bugzilla.redhat.com/show_bug.cgi?id=1853643.
No progress due to higher severity bugs.
From Azure portal console, we could see the event "List Storage Account Keys" has been cached every 5 mins no longer cached several times a minute. Test with 4.6.0-0.nightly-2020-09-10-195619 azure cluster.
And image_registry_operator_azure_key_cache_requests_total metrics has been added image_registry_operator_azure_key_cache_requests_total{endpoint="60000",instance="10.129.0.13:60000",job="image-registry-operator",namespace="openshift-image-registry",pod="cluster-image-registry-operator-f5f4469b4-l8k2f",result="hit",service="image-registry-operator"} 2586 image_registry_operator_azure_key_cache_requests_total{endpoint="60000",instance="10.129.0.13:60000",job="image-registry-operator",namespace="openshift-image-registry",pod="cluster-image-registry-operator-f5f4469b4-l8k2f",result="miss",service="image-registry-operator"} 18