Bug 1906428 - [release-4.6]: When using webhooks in OCP 4.5 fails to rollout latest deploymentconfig
Summary: [release-4.6]: When using webhooks in OCP 4.5 fails to rollout latest deploym...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: openshift-apiserver
Version: 4.6
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.6.z
Assignee: Lukasz Szaszkiewicz
QA Contact: Xingxing Xia
: 1906640 (view as bug list)
Depends On: 1867380
Blocks: 1906429 1909870
TreeView+ depends on / blocked
Reported: 2020-12-10 13:53 UTC by Lukasz Szaszkiewicz
Modified: 2023-09-18 00:23 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Specifies the correct version for the DeploymentConfig. Previously all requests targeting "deploymentconfigs/{name}/instantiate" subresource failed with "no kind DeploymentConfig is registered for version apps.openshift.io/"
Clone Of: 1867380
: 1906429 (view as bug list)
Last Closed: 2021-01-18 17:59:31 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift openshift-apiserver pull 166 0 None closed Bug 1906428: When using webhooks in OCP 4.5 fails to rollout latest deploymentconfig 2021-02-08 10:49:39 UTC
Red Hat Product Errata RHSA-2021:0037 0 None None None 2021-01-18 17:59:57 UTC

Comment 1 Lukasz Szaszkiewicz 2020-12-11 08:27:22 UTC
*** Bug 1906640 has been marked as a duplicate of this bug. ***

Comment 2 Xingxing Xia 2020-12-18 04:54:48 UTC
I'm following JIRA issue DPTP-660 to do the pre-merge verification. Used the cluster-bot launching an env with the still open but Dev-approved PR(s).
The admission-controller.yaml and webhook-configuration.yaml files are copied from https://www.openpolicyagent.org/docs/v0.12.2/kubernetes-admission-control/
For admission-controller.yaml:
Directly creating it, the opa pod will be CrashLoopBackOff for "opa" container with pod logs that show err in binding 443 port. So I must add securityContext field:
         - name: opa
           image: openpolicyagent/opa:0.12.2
             privileged: true
             runAsUser: 0

I must also add SCC (see the step below).

For webhook-configuration.yaml:
I must make some changes (see the step below) otherwise its creation not hit failure, e.g.:
ValidatingWebhookConfiguration is promoted to v1 from v1beta1, so I changed to v1;
And v1 requires sideEffects and admissionReviewVersions must be not empty, so I added them;
And I modified with "operator: In", and "scope: 'Namespaced'", because original link defines too destructively;
And lalbel the test namespace to ensure only the test namespace is affected, and other namespaces in the cluster are not destructed.

Verification steps:
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -days 100000 -out ca.crt -subj "/CN=admission_ca"
cat >server.conf <<EOF
req_extensions = v3_req
distinguished_name = req_distinguished_name
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/CN=opa.opa.svc" -config server.conf
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 100000 -extensions v3_req -extfile server.conf

oc create namespace opa
oc project opa
oc create secret tls opa-server --cert=server.crt --key=server.key
oc adm policy add-scc-to-user privileged -z default
oc apply -f admission-controller.yaml

oc get po
NAME                  READY   STATUS    RESTARTS   AGE
opa-fffdf4574-8pls4   2/2     Running   0          15s

cat > webhook-configuration.yaml <<EOF
kind: ValidatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1
  name: opa-validating-webhook
  - name: validating-webhook.openpolicyagent.org
    - v1beta1
    sideEffects: None
      - key: openpolicyagent.org/webhook
        operator: In
        - ignore
      - operations: ["CREATE", "UPDATE"]
        apiGroups: ["*"]
        apiVersions: ["*"]
        resources: ["*"]
        scope: 'Namespaced'
      caBundle: $(cat ca.crt | base64 | tr -d '\n')
        namespace: opa
        name: opa

oc apply -f webhook-configuration.yaml
oc create ns test-ns
oc label ns test-ns openpolicyagent.org/webhook=ignore
oc create deploymentconfig mydc --image openshift/hello-openshift -n test-ns

oc rollout latest dc/mydc -n test-ns
deploymentconfig.apps.openshift.io/mydc rolled out
######## ^^ oc rollout works well for deploymentconfig without this bug's error

cat > dc-policy.rego << EOF
package kubernetes.admission

deny[msg] {
    input.request.kind.kind == "DeploymentConfig"
    msg:= "No entry for you"

oc create configmap dc-policy --from-file=dc-policy.rego

oc rollout latest dc/mydc -n test-ns
Error from server (No entry for you): admission webhook "validating-webhook.openpolicyagent.org" denied the request: No entry for you
oc create deploymentconfig mydc2 --image openshift/hello-openshift -n test-ns
Error from server (No entry for you): admission webhook "validating-webhook.openpolicyagent.org" denied the request: No entry for you
######## ^^ oc rollout works well for deploymentconfig without this bug's error, and the output is expected as the policy

So the bug is pre-merge verified. After the PR gets merged and bug becomes ON_QA, it will be moved to VERIFIED by the robot automatically (if not, I'll manually move at that time)

Comment 4 Xingxing Xia 2020-12-22 11:24:07 UTC
https://github.com/openshift/openshift-apiserver/pull/166 is merged 14 hours ago, but latest payloads are:
4.6.0-0.nightly-2020-12-21-185524 	Pending 	16 hours ago 			
4.6.0-0.nightly-2020-12-21-163117 	Accepted 	18 hours ago
No payloads contain the PR. So can't move to VERIFIED currently. Waiting for the payload.

Comment 6 Lukasz Szaszkiewicz 2021-01-11 10:43:11 UTC
*** Bug 1909870 has been marked as a duplicate of this bug. ***

Comment 8 errata-xmlrpc 2021-01-18 17:59:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.6.12 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Comment 10 Red Hat Bugzilla 2023-09-18 00:23:53 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days

Note You need to log in before you can comment on or make changes to this bug.