Document URL: https://docs.openshift.com/container-platform/4.2/support/gathering-cluster-data.html AND https://access.redhat.com/documentation/en-us/openshift_container_platform/4.2/html/support/gathering-cluster-data Section Number and Name: 2.2 Describe the issue: In a disconnected env the "oc adm must-gather" command try to download the required image from quay.io and stuck in timeout Suggestions for improvement: --->> MIRROR THE MUST-GATHER IMAGE <<<--- [root@ip-10-0-11-106 openshift]# oc image mirror quay.io/openshift/origin-must-gather:latest ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather:latest -a <your internal and external registries pull secret > ip-10-0-11-106.eu-west-1.compute.internal:5000/ openshift/origin-must-gather blobs: quay.io/openshift/origin-must-gather sha256:be013bd4341b67f7bf9608e9b21c836c228eaeea260ddf8b27b0f5029ea34cca 380B quay.io/openshift/origin-must-gather sha256:bff3b73cbcc496de1de4ea51df88b7249169d0b6eb7d677169eaf90b8a92240e 1.575KiB quay.io/openshift/origin-must-gather sha256:07cfb696f326d7a81a6e209e27540df2c9c1921bdcb732cdf5d998815ff2dff4 2.417KiB quay.io/openshift/origin-must-gather sha256:fff43810c9b02c8e9b64d49dfc82411ed67494949bf53850390109e1b9574061 5.456KiB quay.io/openshift/origin-must-gather sha256:382ffcde118cde096343c044d358f82890b3fc0c7716931f890ecd906856d5a9 7.85MiB quay.io/openshift/origin-must-gather sha256:0c69c6ba9aff58fa90d714941cae5fad040b313b07a584dbfd7beb971cd9661e 23.17MiB quay.io/openshift/origin-must-gather sha256:7b1c937e0f6794db2535be6e4cb6d60a0b668ef78c2576611a3fb9c97a95ccdf 72.71MiB manifests: sha256:dae1257b516a5c177237cfef5a6a3e241962b0d20cf54bcb2b66dc1671c5035e -> latest stats: shared=0 unique=7 size=103.7MiB ratio=1.00 phase 0: ip-10-0-11-106.eu-west-1.compute.internal:5000 openshift/origin-must-gather blobs=7 mounts=0 manifests=1 shared=0 info: Planning completed in 800ms uploading: ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather sha256:7b1c937e0f6794db2535be6e4cb6d60a0b668ef78c2576611a3fb9c97a95ccdf 72.71MiB uploading: ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather sha256:382ffcde118cde096343c044d358f82890b3fc0c7716931f890ecd906856d5a9 7.85MiB uploading: ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather sha256:0c69c6ba9aff58fa90d714941cae5fad040b313b07a584dbfd7beb971cd9661e 23.17MiB sha256:dae1257b516a5c177237cfef5a6a3e241962b0d20cf54bcb2b66dc1671c5035e ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather:latest info: Mirroring completed in 10.03s (10.84MB/s) [root@ip-10-0-11-106 openshift]# --->> USE THE MIRRORED MUST-GATHER IMAGE with the "--image" option <<<--- [root@ip-10-0-60-38 ~]# oc adm must-gather --image=ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather:latest [must-gather ] OUT Using must-gather plugin-in image: ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather:latest [must-gather ] OUT namespace/openshift-must-gather-tlbxv created [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-sxcgw created [must-gather ] OUT pod for plug-in image ip-10-0-11-106.eu-west-1.compute.internal:5000/openshift/origin-must-gather:latest created [must-gather-vwpbp] POD I1030 09:57:20.430045 28 log.go:172] Finished successfully with no errors. [must-gather-vwpbp] POD I1030 09:57:20.585849 37 log.go:172] Gathering data for ns/openshift-cluster-version... ... ... [must-gather-vwpbp] OUT namespaces/openshift/route.openshift.io/routes.yaml [must-gather-vwpbp] OUT [must-gather-vwpbp] OUT sent 38,296 bytes received 2,729,151,581 bytes 49,174,592.38 bytes/sec [must-gather-vwpbp] OUT total size is 2,728,309,842 speedup is 1.00 [must-gather ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-sxcgw deleted [must-gather ] OUT namespace/openshift-must-gather-tlbxv deleted [root@ip-10-0-60-38 ~]# [root@ip-10-0-60-38 ~]# ls -lrt total 0 drwxr-xr-x. 3 root root 161 Oct 30 10:04 must-gather.local.8268045648296743558 [root@ip-10-0-60-38 ~]# Additional information: Mirroring this image is not requested during the internal registry mirror setup and it's not done by the "oc adm release mirror" https://docs.openshift.com/container-platform/4.2/installing/installing_restricted_networks/installing-restricted-networks-preparations.html see also bugzilla 1768185
I added the following note to the procedure: "If your cluster is using a restricted network you must mirror the latest must-gather image and then use it with the --image option. oc image mirror quay.io/openshift/origin-must-gather:latest \ <image destination> \ -a <path to your registry credentials> oc adm must-gather --image=<path to mirrored must-gather image>" @yinzhou can you confirm this is 4.2 and future 4.x versions (4.3, 4.4, etc.)?
Preview is available in the PR here: https://github.com/openshift/openshift-docs/pull/20768
@Kevin Lamenzo `oc image mirror` then use `oc adm must-gather --image` works well , but without mirror , use the default image in the payload also works well , just need to import the image . What's your opinion for command `oc import-image is must-gather -n openshift` then only use `oc adm must-gather`.
Thanks @yinzhou. I added importing the default image as an option. I'm not sure which scenario is better for customers with a disconnected install, so I think including both is safest. Can you help confirm which versions of 4.x this applies to?
Same link for the PR - https://github.com/openshift/openshift-docs/pull/20768/files
(In reply to Kevin Lamenzo from comment #4) > Thanks @yinzhou. I added importing the default image as an option. I'm not > sure which scenario is better for customers with a disconnected install, so > I think including both is safest. > > Can you help confirm which versions of 4.x this applies to? This could apply to 4.1 to latest.
(In reply to zhou ying from comment #6) > (In reply to Kevin Lamenzo from comment #4) > > Thanks @yinzhou. I added importing the default image as an option. I'm not > > sure which scenario is better for customers with a disconnected install, so > > I think including both is safest. > > > > Can you help confirm which versions of 4.x this applies to? > > This could apply to 4.1 to latest. Should be ocp 4.2 to latest. because ocp4.1 not support restricted env.
All the command lgtm in PR, will verify it.
Thanks. Per doc review we're going to take a second look at this. Can you help clarify some things: - First, for the option to import the default must-gather image (oc import-image is/must-gather -n openshift, then oc adm must-gather), do all clusters have this option available, or do they need to perform any prerequisites? Also, is there "build-in" image stream actually named "is" (from "is/must-gather"), or is that something the customer needs to set up. I tested this option on a cluster-bot cluster and it worked, but I don't know how close a cluster-bot install is to a restricted network install. - Second, if the above option works for all (or most clusters), then I think we should document it as the solution. My team member pointed out that clusters in a restricted network will likely not have a connection to the internet and therefore would need to perform additional prerequisites in order to mirror the latest must-gather image. I appreciate your feedback on these few things.
(In reply to Kevin Lamenzo from comment #9) > Thanks. Per doc review we're going to take a second look at this. Can you > help clarify some things: > > - First, for the option to import the default must-gather image (oc > import-image is/must-gather -n openshift, then oc adm must-gather), do all > clusters have this option available, or do they need to perform any > prerequisites? Also, is there "build-in" image stream actually named "is" > (from "is/must-gather"), or is that something the customer needs to set up. > Must be cluster-admin to run the command, 'is' is the short-name of 'image stream' . > I tested this option on a cluster-bot cluster and it worked, but I don't > know how close a cluster-bot install is to a restricted network install. > > - Second, if the above option works for all (or most clusters), then I think > we should document it as the solution. My team member pointed out that > clusters in a restricted network will likely not have a connection to the > internet and therefore would need to perform additional prerequisites in > order to mirror the latest must-gather image. This works out for all cluster include the restricted network, because the default must-gather image from payload . > I appreciate your feedback on these few things.
Thanks. We're going to simplify down to one option then. https://github.com/openshift/openshift-docs/pull/20768 I'll send this to the doc team for review and wrap up once we've published.
This change is live: https://docs.openshift.com/container-platform/4.5/support/gathering-cluster-data.html