Bug 2059210
| Summary: | ingress operator should report Upgradeable False to remind user before upgrade to 4.10 when Non-SAN certs are used | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | OpenShift BugZilla Robot <openshift-bugzilla-robot> |
| Component: | Networking | Assignee: | Miciah Dashiel Butler Masters <mmasters> |
| Networking sub component: | router | QA Contact: | Melvin Joseph <mjoseph> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | urgent | ||
| Priority: | high | CC: | aos-bugs, hongli, mfisher, mmasters, wking, xxia |
| Version: | 4.9 | Keywords: | Upgrades |
| Target Milestone: | --- | ||
| Target Release: | 4.10.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-03-28 12:03:17 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 2057762 | ||
| Bug Blocks: | 2060111 | ||
|
Comment 1
Miciah Dashiel Butler Masters
2022-02-28 16:49:39 UTC
melvinjoseph@mjoseph-mac Downloads % oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False 6m5s Cluster version is 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest
melvinjoseph@mjoseph-mac Downloads % mkdir test_customized_cert_no_san
melvinjoseph@mjoseph-mac Downloads % cd test_customized_cert_no_san
melvinjoseph@mjoseph-mac test_customized_cert_no_san % openssl genrsa -out caKey.pem 2048
Generating RSA private key, 2048 bit long modulus
............................................+++
................................+++
e is 65537 (0x10001)
melvinjoseph@mjoseph-mac test_customized_cert_no_san % openssl req -x509 -new -nodes -key caKey.pem -days 100000 -out caCert.pem -subj "/CN=network_edge_test_ca"
melvinjoseph@mjoseph-mac test_customized_cert_no_san % openssl genrsa -out serverKey.pem 2048
Generating RSA private key, 2048 bit long modulus
...+++
.....................+++
e is 65537 (0x10001)
melvinjoseph@mjoseph-mac test_customized_cert_no_san % cat > server_no_san.conf << EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
EOF
melvinjoseph@mjoseph-mac test_customized_cert_no_san % DOMAIN=$(oc get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')
melvinjoseph@mjoseph-mac test_customized_cert_no_san % $DOMAIN
zsh: command not found: apps.ci-ln-hqg0x12-72292.origin-ci-int-gce.dev.rhcloud.com
melvinjoseph@mjoseph-mac test_customized_cert_no_san % openssl req -new -key serverKey.pem -out serverNoSAN.csr -subj "/CN=*.$DOMAIN" -config server_no_san.conf
melvinjoseph@mjoseph-mac test_customized_cert_no_san % openssl x509 -req -in serverNoSAN.csr -CA caCert.pem -CAkey caKey.pem -CAcreateserial -out serverCertNoSAN.pem -days 100000 -extensions v3_req -extfile server_no_san.conf
Signature ok
subject=/CN=*.apps.ci-ln-hqg0x12-72292.origin-ci-int-gce.dev.rhcloud.com
Getting CA Private Key
melvinjoseph@mjoseph-mac test_customized_cert_no_san % oc get co
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest False False True 19m OAuthServerRouteEndpointAccessibleControllerAvailable: Get "https://oauth-openshift.apps.ci-ln-hqg0x12-72292.origin-ci-int-gce.dev.rhcloud.com/healthz": x509: certificate relies on legacy Common Name field, use SANs instead
baremetal 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 43m
cloud-controller-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 46m
cloud-credential 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 49m
cluster-autoscaler 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 44m
config-operator 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 45m
console 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest False True False 19m DeploymentAvailable: 0 replicas available for console deployment...
csi-snapshot-controller 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 44m
dns 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 44m
etcd 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 42m
image-registry 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 37m
ingress 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 36m
insights 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 38m
kube-apiserver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 40m
kube-controller-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 40m
kube-scheduler 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 38m
kube-storage-version-migrator 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 8m18s
machine-api 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 40m
machine-approver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 43m
machine-config 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 43m
marketplace 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 44m
monitoring 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 30m
network 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 45m
node-tuning 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 4m50s
openshift-apiserver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 9m39s
openshift-controller-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 41m
openshift-samples 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 37m
operator-lifecycle-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 43m
operator-lifecycle-manager-catalog 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 44m
operator-lifecycle-manager-packageserver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 38m
service-ca 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 45m
storage 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 40m
melvinjoseph@mjoseph-mac test_customized_cert_no_san %
melvinjoseph@mjoseph-mac test_customized_cert_no_san % oc get co ingress -o json | jq .status.conditions
[
{
"lastTransitionTime": "2022-03-01T04:36:11Z",
"message": "The \"default\" ingress controller reports Available=True.",
"reason": "IngressAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2022-03-01T04:36:11Z",
"message": "desired and current number of IngressControllers are equal",
"reason": "AsExpected",
"status": "False",
"type": "Progressing"
},
{
"lastTransitionTime": "2022-03-01T04:36:11Z",
"message": "The \"default\" ingress controller reports Degraded=False.",
"reason": "IngressNotDegraded",
"status": "False",
"type": "Degraded"
},
{
"lastTransitionTime": "2022-03-01T04:52:58Z",
"message": "Some ingresscontrollers are not upgradeable: ingresscontroller \"default\" is not upgradeable: OperandsNotUpgradeable: One or more managed resources are not upgradeable: certificate in secret openshift-ingress/custom-certs-default has legacy Common Name (CN) but has no Subject Alternative Name (SAN) for domain: *.apps.ci-ln-hqg0x12-72292.origin-ci-int-gce.dev.rhcloud.com",
"reason": "IngressControllersNotUpgradeable",
"status": "False",
"type": "Upgradeable"
}
]
VERIFIED THE CLUSTER NOT UPGRADE-ABLE.
melvinjoseph@mjoseph-mac Downloads % mkdir tmp_dir
melvinjoseph@mjoseph-mac Downloads % cd tmp_dir
melvinjoseph@mjoseph-mac tmp_dir % curl -O -sS https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/ca.key
melvinjoseph@mjoseph-mac tmp_dir % curl -O -sS https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/ca.pem
melvinjoseph@mjoseph-mac tmp_dir % curl -O -sS https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/openssl.conf
melvinjoseph@mjoseph-mac tmp_dir % sed -i.bak "s/example.com/${domain}/g" openssl.conf
melvinjoseph@mjoseph-mac tmp_dir % openssl genrsa -out apps.key 2048
Generating RSA private key, 2048 bit long modulus
..............................................................+++
.........................+++
e is 65537 (0x10001)
melvinjoseph@mjoseph-mac tmp_dir % openssl req -new -config openssl.conf -key apps.key -out apps.csr
melvinjoseph@mjoseph-mac tmp_dir % openssl x509 -req -CA ca.pem -CAkey ca.key -CAcreateserial -extfile openssl.conf -extensions v3_req -in apps.csr -out apps.crt -days 3650
Signature ok
subject=/C=US/ST=VA/L=Somewhere/O=RedHat/OU=OpenShift QE/CN=apps
Getting CA Private Key
melvinjoseph@mjoseph-mac tmp_dir % openssl x509 -text -noout -in apps.crt | grep "Alternative Name" -A 1
X509v3 Subject Alternative Name:
DNS:*.apps.ci-ln-hqg0x12-72292.origin-ci-int-gce.dev.rhcloud.com
melvinjoseph@mjoseph-mac Downloads % oc --namespace openshift-ingress create secret tls custom-certs-default-new --cert=tmp_dir/apps.crt --key=tmp_dir/apps.key
secret/custom-certs-default-new created
melvinjoseph@mjoseph-mac Downloads % oc patch --type=merge --namespace openshift-ingress-operator ingresscontrollers/default \
--patch '{"spec":{"defaultCertificate":{"name":"custom-certs-default-new"}}}'
ingresscontroller.operator.openshift.io/default patched
melvinjoseph@mjoseph-mac Downloads % oc create configmap user-ca-bundle2 --from-file=ca-bundle.crt=tmp_dir/ca.pem -n openshift-config
configmap/user-ca-bundle2 created
melvinjoseph@mjoseph-mac Downloads % oc patch proxy/cluster --patch '{"spec":{"trustedCA":{"name":"user-ca-bundle2"}}}' --type=merge
proxy.config.openshift.io/cluster patched
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get secret
NAME TYPE DATA AGE
builder-dockercfg-wxc6v kubernetes.io/dockercfg 1 128m
builder-token-clzcl kubernetes.io/service-account-token 4 129m
builder-token-lj768 kubernetes.io/service-account-token 4 128m
custom-certs-default kubernetes.io/tls 2 107m
custom-certs-default-new kubernetes.io/tls 2 71s
custom-certs-default-san kubernetes.io/tls 2 78m
default-dockercfg-9gxml kubernetes.io/dockercfg 1 129m
default-token-rtmwm kubernetes.io/service-account-token 4 129m
default-token-zg8zs kubernetes.io/service-account-token 4 131m
deployer-dockercfg-k7gwj kubernetes.io/dockercfg 1 128m
deployer-token-cnwn2 kubernetes.io/service-account-token 4 129m
deployer-token-cqkj9 kubernetes.io/service-account-token 4 128m
router-dockercfg-lnqlc kubernetes.io/dockercfg 1 129m
router-metrics-certs-default kubernetes.io/tls 2 131m
router-stats-default Opaque 2 131m
router-token-hzsgp kubernetes.io/service-account-token 4 129m
router-token-tvff5 kubernetes.io/service-account-token 4 131m
melvinjoseph@mjoseph-mac Downloads % oc get co
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 15m
baremetal 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 146m
cloud-controller-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 149m
cloud-credential 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 152m
cluster-autoscaler 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 147m
config-operator 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 148m
console 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 15m
csi-snapshot-controller 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 147m
dns 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 147m
etcd 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 145m
image-registry 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 140m
ingress 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 139m
insights 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 141m
kube-apiserver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 144m
kube-controller-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 143m
kube-scheduler 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 141m
kube-storage-version-migrator 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 10m
machine-api 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 143m
machine-approver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 146m
machine-config 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 146m
marketplace 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 147m
monitoring 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 133m
network 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 148m
node-tuning 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 84m
openshift-apiserver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 112m
openshift-controller-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 91m
openshift-samples 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 141m
operator-lifecycle-manager 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 146m
operator-lifecycle-manager-catalog 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 147m
operator-lifecycle-manager-packageserver 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 141m
service-ca 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 148m
storage 4.10.0-0.ci.test-2022-03-01-041438-ci-ln-hqg0x12-latest True False False 143m
melvinjoseph@mjoseph-mac Downloads % oc get co ingress -o json | jq .status.conditions
[
{
"lastTransitionTime": "2022-03-01T04:36:11Z",
"message": "The \"default\" ingress controller reports Available=True.",
"reason": "IngressAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2022-03-01T04:36:11Z",
"message": "desired and current number of IngressControllers are equal",
"reason": "AsExpected",
"status": "False",
"type": "Progressing"
},
{
"lastTransitionTime": "2022-03-01T04:36:11Z",
"message": "The \"default\" ingress controller reports Degraded=False.",
"reason": "IngressNotDegraded",
"status": "False",
"type": "Degraded"
},
{
"lastTransitionTime": "2022-03-01T05:22:10Z",
"reason": "IngressControllersUpgradeable",
"status": "True",
"type": "Upgradeable"
}
]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.10.6 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:1026 I am setting "No Doc Update" on this BZ because the only reason for backporting the change to 4.10 was so that we could backport it to 4.9. I am adding appropriate doc text to the 4.9 BZ, bug 2060111. |