Bug 1906429 - [release-4.5]: When using webhooks in OCP 4.5 fails to rollout latest deploymentconfig
Summary: [release-4.5]: When using webhooks in OCP 4.5 fails to rollout latest deploym...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: openshift-apiserver
Version: 4.5
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.5.z
Assignee: Lukasz Szaszkiewicz
QA Contact: Xingxing Xia
: 1907958 1909870 (view as bug list)
Depends On: 1906428
TreeView+ depends on / blocked
Reported: 2020-12-10 13:55 UTC by Lukasz Szaszkiewicz
Modified: 2021-01-26 16:32 UTC (History)
17 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Specifies the correct version for the DeploymentConfig. Previously all requests targeting "deploymentconfigs/{name}/instantiate" subresource failed with "no kind DeploymentConfig is registered for version apps.openshift.io/"
Clone Of: 1906428
Last Closed: 2021-01-26 16:32:13 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift openshift-apiserver pull 168 0 None closed Bug 1906429: When using webhooks in OCP 4.5 fails to rollout latest deploymentconfig 2021-02-08 10:49:58 UTC
Red Hat Product Errata RHBA-2021:0175 0 None None None 2021-01-26 16:32:19 UTC

Comment 1 Stefan Schimanski 2020-12-15 15:16:03 UTC
*** Bug 1907958 has been marked as a duplicate of this bug. ***

Comment 3 Xingxing Xia 2020-12-18 08:35:12 UTC
openshift/openshift-apiserver/pull/168 is still open, the bug should not be ON_QA, moving it back to POST.
Per the robot's workflow, 4.5 PR cannot be merged if the 4.6 bug isn't VERIFIED, 4.6 bug cannot be VERIFIED if the 4.6 PR isn't merged.
Now 4.6 PR is still open. Good news is I already pre-merge verified the 4.6 bug https://bugzilla.redhat.com/show_bug.cgi?id=1906428#c2 . Waiting for 4.6 PR's merging.

Comment 4 Lukasz Szaszkiewicz 2021-01-11 13:15:10 UTC
*** Bug 1909870 has been marked as a duplicate of this bug. ***

Comment 5 Xingxing Xia 2021-01-12 03:43:07 UTC
I'm following JIRA issue DPTP-660 to do the pre-merge verification. Used the cluster-bot launching an env with the still open but Dev-approved PR(s): launch openshift/openshift-apiserver#168

The admission-controller.yaml and webhook-configuration.yaml files are copied from https://www.openpolicyagent.org/docs/v0.12.2/kubernetes-admission-control/ . For webhook-configuration.yaml:
  I must make some changes (see the step below) otherwise its creation not hit failure, e.g.:
  ValidatingWebhookConfiguration is promoted to v1 from v1beta1, so I changed to v1;
  And v1 requires sideEffects and admissionReviewVersions must be not empty, so I added them;
  And I modified with "operator: In", and "scope: 'Namespaced'", because original link defines too destructively;
  And lalbel the test namespace to ensure only the test namespace is affected, and other namespaces in the cluster are not destructed.

Verification steps:
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -days 100000 -out ca.crt -subj "/CN=admission_ca"
cat >server.conf <<EOF
req_extensions = v3_req
distinguished_name = req_distinguished_name
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/CN=opa.opa.svc" -config server.conf
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 100000 -extensions v3_req -extfile server.conf

oc create namespace opa
oc project opa
oc create secret tls opa-server --cert=server.crt --key=server.key
oc adm policy add-scc-to-user anyuid -z default

oc apply -f admission-controller.yaml
oc get po
NAME                   READY   STATUS    RESTARTS   AGE
opa-75c7957bb7-dh9l8   2/2     Running   0          54s

cat > webhook-configuration.yaml <<EOF
kind: ValidatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1
  name: opa-validating-webhook
  - name: validating-webhook.openpolicyagent.org
    - v1beta1
    sideEffects: None
      - key: openpolicyagent.org/webhook
        operator: In
        - ignore
      - operations: ["CREATE", "UPDATE"]
        apiGroups: ["*"]
        apiVersions: ["*"]
        resources: ["*"]
        scope: 'Namespaced'
      caBundle: $(cat ca.crt | base64 | tr -d '\n')
        namespace: opa
        name: opa

oc apply -f webhook-configuration.yaml

oc create ns test-ns
oc label ns test-ns openpolicyagent.org/webhook=ignore
oc create deploymentconfig mydc --image openshift/hello-openshift -n test-ns

oc rollout latest dc/mydc -n test-ns
deploymentconfig.apps.openshift.io/mydc rolled out
######## ^^ oc rollout works well for deploymentconfig without this bug's error

cat > dc-policy.rego << EOF
package kubernetes.admission

deny[msg] {
    input.request.kind.kind == "DeploymentConfig"
    msg:= "No entry for you"

oc create configmap dc-policy --from-file=dc-policy.rego
oc rollout latest dc/mydc -n test-ns
Error from server (No entry for you): admission webhook "validating-webhook.openpolicyagent.org" denied the request: No entry for you

oc create deploymentconfig mydc2 --image openshift/hello-openshift -n test-ns
Error from server (No entry for you): admission webhook "validating-webhook.openpolicyagent.org" denied the request: No entry for you
######## ^^ oc rollout works well for deploymentconfig without this bug's error, and the output is expected as the policy

So the bug is pre-merge verified. After the PR gets merged and bug becomes ON_QA, it will be moved to VERIFIED by the robot automatically (if not, I'll manually move at that time)

Comment 6 Lukasz Szaszkiewicz 2021-01-15 10:13:11 UTC
PR is in the merge queue.

Comment 10 errata-xmlrpc 2021-01-26 16:32:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.5.28 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.