--- Additional comment from Michael Gugino on 2021-03-19 13:47:55 UTC --- --snip-- https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.8-e2e-vsphere/1372795001506893824 vsphere test has one worker node fail to issue a CSR. The machine was created, 3 minutes later the machine was 'provisioned' indicating the cloud was running the instance. However, this cluster has 5 nodes (2 workers, 3 masters) but 15 approved CSRs. There should only be 10 approved CSRs with 5 nodes (1 client, 1 server). This indicates there's some problem with the hosts or the CSR approver. ---- New info: One worker failed to join the cluster. Looking at another kubelet's logs, it's completely spammed with the following entries. This looks significant. Mar 19 07:42:23.567191 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 hyperkube[1728]: E0319 07:42:23.566944 1728 kubelet_volumes.go:154] orphaned pod "043e804b-7102-4832-8dcd-362d5d10c9fe" found, but volume paths are still present on disk : There were a total of 39 errors similar to this. Turn up verbosity to see them. Mar 19 07:42:23.959907 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:23.970798 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:25.373491 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:25.386088 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:25.568172 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 hyperkube[1728]: E0319 07:42:25.568120 1728 kubelet_volumes.go:154] orphaned pod "043e804b-7102-4832-8dcd-362d5d10c9fe" found, but volume paths are still present on disk : There were a total of 39 errors similar to this. Turn up verbosity to see them. Mar 19 07:42:26.296502 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:26.307259 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:26.622206 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:26.633790 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:27.027446 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:27.038595 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:27.257966 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:27.274212 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument Mar 19 07:42:27.361240 ci-op-5dk64ln5-36a8b-9zsjg-worker-7knv5 systemd[1]: Failed to set up mount unit: Invalid argument
*** This bug has been marked as a duplicate of bug 1884800 ***