test: [sig-builds][Feature:Builds] verify /run filesystem contents do not have unexpected content using a simple Docker Strategy Build is failing frequently in CI, see search results: https://search.ci.openshift.org/?maxAge=168h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job&search=%5C%5Bsig-builds%5C%5D%5C%5BFeature%3ABuilds%5C%5D+verify+%2Frun+filesystem+contents++do+not+have+unexpected+content+using+a+simple+Docker+Strategy+Build Failing job example: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ocp-4.7-e2e-aws-workers-rhel7/1356580635287752704 Log snippet: =================================================================== [AfterEach] [sig-builds][Feature:Builds] verify /run filesystem contents github.com/openshift/origin/test/extended/util/client.go:138 STEP: Collecting events from namespace "e2e-test-verify-run-fs-bk6kv". STEP: Found 11 events. Feb 2 13:46:33.787: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for verify-run-fs-1-build: { } Scheduled: Successfully assigned e2e-test-verify-run-fs-bk6kv/verify-run-fs-1-build to ip-10-0-139-144.us-west-2.compute.internal Feb 2 13:46:33.787: INFO: At 2021-02-02 13:44:46 +0000 UTC - event for e2e-test-verify-run-fs-bk6kv: {namespace-security-allocation-controller } CreatedSCCRanges: created SCC ranges Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:07 +0000 UTC - event for verify-run-fs-1-build: {multus } AddedInterface: Add eth0 [10.128.5.136/23] Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:10 +0000 UTC - event for verify-run-fs-1-build: {kubelet ip-10-0-139-144.us-west-2.compute.internal} Pulled: Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dad6be8a28ea564c1d4794981b3c7a090339be8141d0bb3d564948cec097f4e4" already present on machine Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:12 +0000 UTC - event for verify-run-fs-1-build: {kubelet ip-10-0-139-144.us-west-2.compute.internal} Created: Created container manage-dockerfile Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:12 +0000 UTC - event for verify-run-fs-1-build: {kubelet ip-10-0-139-144.us-west-2.compute.internal} Started: Started container manage-dockerfile Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:13 +0000 UTC - event for verify-run-fs-1-build: {kubelet ip-10-0-139-144.us-west-2.compute.internal} Pulled: Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dad6be8a28ea564c1d4794981b3c7a090339be8141d0bb3d564948cec097f4e4" already present on machine Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:14 +0000 UTC - event for verify-run-fs-1-build: {kubelet ip-10-0-139-144.us-west-2.compute.internal} Created: Created container docker-build Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:14 +0000 UTC - event for verify-run-fs-1-build: {kubelet ip-10-0-139-144.us-west-2.compute.internal} Started: Started container docker-build Feb 2 13:46:33.787: INFO: At 2021-02-02 13:45:15 +0000 UTC - event for verify-run-fs-1: {build-controller } BuildStarted: Build e2e-test-verify-run-fs-bk6kv/verify-run-fs-1 is now running Feb 2 13:46:33.787: INFO: At 2021-02-02 13:46:27 +0000 UTC - event for verify-run-fs-1: {build-controller } BuildCompleted: Build e2e-test-verify-run-fs-bk6kv/verify-run-fs-1 completed successfully Feb 2 13:46:33.877: INFO: POD NODE PHASE GRACE CONDITIONS Feb 2 13:46:33.877: INFO: verify-run-fs-1-build ip-10-0-139-144.us-west-2.compute.internal Succeeded [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-02-02 13:45:13 +0000 UTC PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-02-02 13:46:27 +0000 UTC PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-02-02 13:46:27 +0000 UTC PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-02-02 13:45:03 +0000 UTC }] Feb 2 13:46:33.877: INFO: Feb 2 13:46:34.047: INFO: skipping dumping cluster info - cluster too large Feb 2 13:46:34.143: INFO: Deleted {user.openshift.io/v1, Resource=users e2e-test-verify-run-fs-bk6kv-user}, err: <nil> Feb 2 13:46:34.237: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthclients e2e-client-e2e-test-verify-run-fs-bk6kv}, err: <nil> Feb 2 13:46:34.332: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens y75FKEWkTiuxePbL0_mrIAAAAAAAAAAA}, err: <nil> [AfterEach] [sig-builds][Feature:Builds] verify /run filesystem contents github.com/openshift/origin/test/extended/util/client.go:139 Feb 2 13:46:34.332: INFO: Waiting up to 7m0s for all (but 100) nodes to be ready STEP: Destroying namespace "e2e-test-verify-run-fs-bk6kv" for this suite. Feb 2 13:46:34.518: INFO: Running AfterSuite actions on all nodes Feb 2 13:46:34.518: INFO: Running AfterSuite actions on node 1 fail [github.com/openshift/origin/test/extended/builds/run_fs_verification.go:120]: Expected <bool>: false to be true
Looks like for RHEL7 workers the structure of /run/secrets/rhsm is slightly different: ``` /run: lock rhsm secrets /run/lock: /run/rhsm: /run/secrets: rhsm /run/secrets/rhsm: ca logging.conf rhsm.conf syspurpose /run/secrets/rhsm/ca: redhat-uep.pem ```
@adam.kaplan, Since the PR already merged and is in the release-4.7 branch, can this be moved back to 4.7?
Yes codes merged in .47 and passed in 4.7:https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.7/1357977718846656512 @adam if need to move the bug to 4.7 like Comment 4
FYI pass in search:https://search.ci.openshift.org/?search=%5C%5Bsig-builds%5C%5D%5C%5BFeature%3ABuilds%5C%5D+verify+%2Frun+filesystem+contents++do+not+have+unexpected+content+using+a+simple+Docker+Strategy+Build&maxAge=12h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job release-openshift-ocp-installer-e2e-aws-mirrors-4.7 error is no related to the bug
Moving back to 4.7.0
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633