Hide Forgot
Description of problem: Unless the user is aware that .openshift_install_state.json exists, they often produce ignition files which are already expired due to stale certificates being pulled in from the file. This results in failed installations. Version-Release number of the following components: 4.5.5 How reproducible: Consistently Steps to Reproduce: 1. Create ignition files using the installer 2. Recreate ignition files and the certificates from the first creation are injected in to the ignition files 3. Actual results: Installer creates expired ignition content Expected results: Installer should warn the user about the expired content or fail to proceed with a warning Additional info: Checked certs befor and after with: FILES=`jq '.storage.files[].path' bootstrap.ign -r | grep crt` for FILE in $FILES; do echo $FILE; jq ".storage.files[] | select(.path==\"$FILE\") | .contents.source" bootstrap.ign -r | sed "s/data\:text\/plain\;charset\=utf\-8;base64,//g" | base64 -d | openssl x509 -text -noout | grep "Not Before\: \| Not After \: \| $FILE"; done
This is the current known behaviour of the installer that it provides you the same assets from the state file as long as it exists.. it does not generate new ones until the inputs change.
Understood, the problem is that users are unknowingly creating expired certs and not realizing it until after support has analyzed the installer-gather. It is not common knowledge that the installer uses cached content. It would seem that the installer could detect this condition and provide a warning.
> It would seem that the installer could detect this condition and provide a warning. Agree. A warning message is helpful for users to understand there maybe risk if old state file exists. Hope we could get this addressed in 4.6, thanks.
(In reply to Gaoyun Pei from comment #3) > > It would seem that the installer could detect this condition and provide a warning. > Agree. A warning message is helpful for users to understand there maybe risk > if old state file exists. > Hope we could get this addressed in 4.6, thanks. The current behaviour of the state file has been in place since the start and therefore we expect the users already have the right expectations. Changing the installer to allow validating and warning existing assets would need change in the core code-path and we can't fix it at this stage in 4.6 cycle. So we need to move it to 4.7 to work on it.
This is by definition not a 4.6 blocker because it's not a regression and has been this way since GA. Moving to 4.7.0, we can evaluate backporting to 4.6.z once there's an agreed upon fix.
PR: https://github.com/openshift/installer/pull/4287
Verify this bug with openshift-install 4.7.0-0.nightly-2020-10-24-155529 Steps: 1. Create ignition files using the installer ./openshift-install create ignition-configs --dir gpei-01 2. Change the system time to several days later 3. Re-create ignition files in the same folder # ./openshift-install create ignition-configs --dir gpei-01 WARNING Bootstrap Ignition-Config Certificate aggregator-ca.crt expired at 2020-10-27T08:07:29Z. WARNING Bootstrap Ignition-Config Certificate aggregator-ca-bundle.crt expired at 2020-10-27T08:07:29Z. WARNING Bootstrap Ignition-Config Certificate aggregator-client.crt expired at 2020-10-27T08:07:30Z. WARNING Bootstrap Ignition-Config Certificate aggregator-signer.crt expired at 2020-10-27T08:07:29Z. WARNING Bootstrap Ignition-Config Certificate apiserver-proxy.crt expired at 2020-10-27T08:07:30Z. WARNING Bootstrap Ignition-Config Certificate kube-apiserver-lb-server.crt expired at 2020-10-27T08:07:31Z. WARNING Bootstrap Ignition-Config Certificate kube-apiserver-internal-lb-server.crt expired at 2020-10-27T08:07:31Z. WARNING Bootstrap Ignition-Config Certificate kube-apiserver-localhost-server.crt expired at 2020-10-27T08:07:31Z. WARNING Bootstrap Ignition-Config Certificate kube-apiserver-service-network-server.crt expired at 2020-10-27T08:07:31Z. WARNING Bootstrap Ignition-Config Certificate kube-apiserver-complete-client-ca-bundle.crt expired at 2020-10-27T08:07:32Z. WARNING Bootstrap Ignition-Config Certificate kubelet-client-ca-bundle.crt expired at 2020-10-27T08:07:32Z. WARNING Bootstrap Ignition-Config Certificate kubelet-signer.crt expired at 2020-10-27T08:07:32Z. WARNING Bootstrap Ignition-Config Certificate kubelet-serving-ca-bundle.crt expired at 2020-10-27T08:07:32Z. WARNING Bootstrap Ignition-Config: 13 certificates expired. Installation attempts with the created Ignition-Configs will possibly fail. INFO Ignition-Configs created in: gpei-01 and gpei-01/auth
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633
Thanks to whoever posted this. This is has and will catch people out causing a really bad experience of openshift. Its caching a dumb thing to do specially when certificates expire in 24hrs. Whoever wrote the code that cashes content and certificates needs to be supervised. This is a basic 101 mistake.