Bug 2008713 - VPA webhook timeout prevents all pods from starting
Summary: VPA webhook timeout prevents all pods from starting
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.7
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.9.z
Assignee: Joel Smith
QA Contact: Weinan Liu
URL:
Whiteboard:
Depends On: 2008712
Blocks: 2083758
TreeView+ depends on / blocked
 
Reported: 2021-09-29 00:36 UTC by Joel Smith
Modified: 2023-09-15 01:36 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 2008712
Environment:
Last Closed: 2022-05-03 07:35:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift vertical-pod-autoscaler-operator pull 109 0 None open [release-4.9] Bugzilla 2008712: Set VPA Webhook timeout to 10s to match v1 webhook default 2021-09-29 00:36:23 UTC
Red Hat Product Errata RHBA-2022:1605 0 None None None 2022-05-03 07:35:52 UTC

Description Joel Smith 2021-09-29 00:36:24 UTC
+++ This bug was initially created as a clone of Bug #2008712 +++

Description of problem:
When VPA is installed, if its webhook service becomes slow or unreachable, the API server should ignore the failure and instead continue with pod creation.

The VPA webhook has such a long timeout that if the webhook service is in a bad state, the entire pod creation request fails without proceeding beyond the VPA webhook timeout. 

How reproducible:
100%

Steps to Reproduce:
1. Install VPA
2. oc get deployment -n openshift-vertical-pod-autoscaler vpa-admission-plugin-default -o jsonpath='{.spec.template.spec.containers[0].args}' | jq
3. oc get mutatingwebhookconfiguration vpa-webhook-config -o jsonpath='{.webhooks[0].timeoutSeconds}{"\n"}'

Actual results:
[
  "--logtostderr",
  "--v=1",
  "--tls-cert-file=/data/tls-certs/tls.crt",
  "--tls-private-key=/data/tls-certs/tls.key",
  "--client-ca-file=/data/tls-ca-certs/service-ca.crt"
]
30

Expected results:
[
  "--logtostderr",
  "--v=1",
  "--tls-cert-file=/data/tls-certs/tls.crt",
  "--tls-private-key=/data/tls-certs/tls.key",
  "--client-ca-file=/data/tls-ca-certs/service-ca.crt"
  "--webhook-timeout-seconds=10"
]
10

Comment 4 Weinan Liu 2022-04-29 08:45:47 UTC
$ oc get deployment -n openshift-vertical-pod-autoscaler vpa-admission-plugin-default -o jsonpath='{.spec.template.spec.containers[0].args}' | jq
[
  "--logtostderr",
  "--v=1",
  "--tls-cert-file=/data/tls-certs/tls.crt",
  "--tls-private-key=/data/tls-certs/tls.key",
  "--client-ca-file=/data/tls-ca-certs/service-ca.crt",
  "--webhook-timeout-seconds=10"
]
$ oc get mutatingwebhookconfiguration vpa-webhook-config -o jsonpath='{.webhooks[0].timeoutSeconds}{"\n"}'
10

ose-vertical-pod-autoscaler-operator-metadata-container-v4.9.0.202204220627

Comment 6 errata-xmlrpc 2022-05-03 07:35:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.31 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1605

Comment 7 Red Hat Bugzilla 2023-09-15 01:36:16 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days


Note You need to log in before you can comment on or make changes to this bug.