The current wait interval in case of failed upload due to HTTP 403 response is (2h,8h) given the default gathering interval is 2h. I think it makes sense to make this interval shorter.
Verified on 4.10.0-0.ci-2021-11-30-141534. Verification steps: 1. Modify/Create support secret with the following keys/values: endpoint: https://httpbin.org/status/403 interval: 5m 2. Restart insights operator 3. Verify insights operator logs and search for the following: "Configuration updated: enabled=true endpoint=https://httpbin.org/status/403 interval=5m0s username=false token=true reportEndpoint=https://cloud.redhat.com/api/insights-results-aggregator/v1/clusters/%s/report initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s"
Correction: Verified on 4.10.0-0.ci-2021-12-01-042705. Verification steps: 1. Modify/Create support secret with the following keys/values: namespace: openshift-config endpoint: https://httpbin.org/status/403 interval: 5m 2. Restart insights operator 3. Verify insights operator logs and search for the following: "I1201 13:35:01.887203 1 insightsclient.go:269] gateway server https://httpbin.org/status/403 returned 403, x-rh-insights-request-id=" ... "I1201 13:39:09.113525 1 insightsclient.go:269] gateway server https://httpbin.org/status/403 returned 403, x-rh-insights-request-id=" ... "I1201 13:45:09.631824 1 insightsclient.go:269] gateway server https://httpbin.org/status/403 returned 403, x-rh-insights-request-id=" There will be 5 attempts before being marked as degraded. The time between attempts has to be between [interval/2, interval/2 + 2 * interval/2] In our case, [2.5m, 7.5m]
Correction: In step 1, the namespace should be openshift-config but it is not a key/value.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056