Bug 1955194 - ocp disconnected mode machine config pool mark as degraded
Summary: ocp disconnected mode machine config pool mark as degraded
Keywords:
Status: CLOSED DUPLICATE of bug 1950112
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Machine Config Operator
Version: 4.8
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: ---
Assignee: Yu Qi Zhang
QA Contact: Michael Nguyen
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-29 16:25 UTC by Sebastian Scheinkman
Modified: 2021-05-03 17:10 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-03 17:10:01 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift machine-config-operator pull 2553 0 None Waiting on Customer Some pods are constantly being rescheduled/restarted 2022-05-03 02:36:21 UTC

Description Sebastian Scheinkman 2021-04-29 16:25:46 UTC
Description of problem:

on disconnected mode with ImageContentSourcePolicy and mirror the nodes mode to degraded when try to install rt kernel.

logs from the machine config daemon

```
I0429 14:53:05.046167  752470 run.go:18] Running: nice -- ionice -c 3 oc image extract --path /:/run/mco-machine-os-content/os-content-215884633 --registry-config /var/lib/kubelet/config.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:826588080c4a98587f2cb45be94d9f162df2cfb3e988dd1de2e4d49de7a96aa3
error: unable to connect to image repository quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:826588080c4a98587f2cb45be94d9f162df2cfb3e988dd1de2e4d49de7a96aa3: Get "https://quay.io/v2/": dial tcp 34.224.196.162:443: connect: network is unreachable
W0429 14:53:05.101961  752470 run.go:44] nice failed: running nice -- ionice -c 3 oc image extract --path /:/run/mco-machine-os-content/os-content-215884633 --registry-config /var/lib/kubelet/config.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:826588080c4a98587f2cb45be94d9f162df2cfb3e988dd1de2e4d49de7a96aa3 failed: error: unable to connect to image repository quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:826588080c4a98587f2cb45be94d9f162df2cfb3e988dd1de2e4d49de7a96aa3: Get "https://quay.io/v2/": dial tcp 34.224.196.162:443: connect: network is unreachable
: exit status 1; retrying...
I0429 14:53:05.101990  752470 update.go:314] Falling back to using podman cp to fetch OS image content
I0429 14:53:05.102021  752470 run.go:18] Running: nice -- ionice -c 3 podman pull -q --authfile /var/lib/kubelet/config.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:826588080c4a98587f2cb45be94d9f162df2cfb3e988dd1de2e4d49de7a96aa3
d341d1cabe18c3ca6b1295549c2b81a09afa0bbddaeb5bde42ee72605760d77a
I0429 14:53:05.330641  752470 rpm-ostree.go:258] Running captured: podman create --net=none --annotation=org.openshift.machineconfigoperator.pivot=true --name ostree-container-pivot-77405bf3-4e66-44b3-a93f-e6bc70dd8cc1 quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:826588080c4a98587f2cb45be94d9f162df2cfb3e988dd1de2e4d49de7a96aa3
I0429 14:53:05.496648  752470 run.go:18] Running: nice -- ionice -c 3 podman cp 770bc9d09df714fc17edb14ad108ab932855346abd01240161567b33dfa548b1:/ /run/mco-machine-os-content/os-content-215884633
I0429 14:53:10.086698  752470 rpm-ostree.go:258] Running captured: chcon -R -t var_run_t /run/mco-machine-os-content/os-content-215884633
I0429 14:53:10.087770  752470 update.go:278] Error changing selinux context on path /run/mco-machine-os-content/os-content-215884633  error running chcon -R -t var_run_t /run/mco-machine-os-content/os-content-215884633: chcon: cannot access '/run/mco-machine-os-content/os-content-215884633': No such file or directory

```

The problem is when the daemon tries to fallback to podman the cp command doesn't create the folder for the os-content an put the files under the parent folder then the next step for selinux fails

Error changing selinux context on path /run/mco-machine-os-content/os-content-215884633  error running chcon -R -t var_run_t /run/mco-machine-os-content/os-content-215884633: chcon: cannot access '/run/mco-machine-os-content/os-content-215884633': No such file or directory

Comment 1 Yu Qi Zhang 2021-04-29 16:42:07 UTC
This should be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1950112, are you still seeing this in the latest nightlies?

Comment 2 Sebastian Scheinkman 2021-04-29 17:00:47 UTC
Hi Yu,

thanks for the comment I am running

Client Version: 4.8.0-0.nightly-2021-04-18-203506
Server Version: 4.8.0-0.nightly-2021-04-18-203506
Kubernetes Version: v1.21.0-rc.0+2993be8


I will try to redeploy my environment and check it again

Thanks!
Sebastian

Comment 3 Yu Qi Zhang 2021-05-03 17:10:01 UTC

*** This bug has been marked as a duplicate of bug 1950112 ***


Note You need to log in before you can comment on or make changes to this bug.