Bug 1744456 - [disconnected] rhel node worker can not be installed in a disconnected env
Summary: [disconnected] rhel node worker can not be installed in a disconnected env
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.2.0
Assignee: Russell Teague
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-08-22 08:30 UTC by Johnny Liu
Modified: 2019-10-16 06:37 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-16 06:37:16 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift openshift-ansible pull 11858 0 'None' closed Bug 1744456: Extract registries.conf from bootstrap 2021-01-06 01:54:59 UTC
Red Hat Product Errata RHBA-2019:2922 0 None None None 2019-10-16 06:37:27 UTC

Description Johnny Liu 2019-08-22 08:30:16 UTC
Description of problem:

Version-Release number of the following components:
master branch of openshift-ansible

How reproducible:
Always

Steps to Reproduce:
1. Ensure your network does not have internet connectivity
2. Mirror release image to local registry
3. Add 'imageContentSources' and local registry CA into install-config.yaml
4. Trigger a UPI install on baremetal
5. Scale up rhel worker node

Actual results:
playbook is trying to pull release image from internet.

TASK [openshift_node : Pull release image] *******************************************************************************************************************
fatal: [10.0.151.63]: FAILED! => {"changed": true, "cmd": ["podman", "pull", "--tls-verify=False", "--authfile", "/tmp/ansible.mEQ2RD/pull-secret.json", "registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26"], "delta": "0:02:00.494737", "end": "2019-08-22 04:09:27.378653", "msg": "non-zero return code", "rc": 125, "start": "2019-08-22 04:07:26.883916", "stderr": "Trying to pull registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26...time=\"2019-08-22T04:09:27-04:00\" level=error msg=\"Error pulling image ref //registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: Error initializing source docker://registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: pinging docker registry returned: Get http://registry.svc.ci.openshift.org/v2/: dial tcp 35.196.103.194:80: i/o timeout\"\nFailed\nError: error pulling image \"registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26\": unable to pull registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: unable to pull image: Error initializing source docker://registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: pinging docker registry returned: Get http://registry.svc.ci.openshift.org/v2/: dial tcp 35.196.103.194:80: i/o timeout", "stderr_lines": ["Trying to pull registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26...time=\"2019-08-22T04:09:27-04:00\" level=error msg=\"Error pulling image ref //registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: Error initializing source docker://registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: pinging docker registry returned: Get http://registry.svc.ci.openshift.org/v2/: dial tcp 35.196.103.194:80: i/o timeout\"", "Failed", "Error: error pulling image \"registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26\": unable to pull registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: unable to pull image: Error initializing source docker://registry.svc.ci.openshift.org/ocp/release@sha256:55b073d15bf225317cf6c04eaddb528b5d056d75eb8ffb2b7937cbfc67d4ed26: pinging docker registry returned: Get http://registry.svc.ci.openshift.org/v2/: dial tcp 35.196.103.194:80: i/o timeout"], "stdout": "", "stdout_lines": []}

Expected results:
playbook should be able to scale up rhel worker node in disconnected install.

Additional info:

Comment 3 Johnny Liu 2019-08-30 06:24:23 UTC
Verified this bug with openshift-ansible-4.2.0-201908281019.git.195.01175db.el7, and PASS.

[root@rhel-0 ~]# ping www.google.com -c 3
PING www.google.com (172.217.13.228) 56(84) bytes of data.

--- www.google.com ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2000ms

[root@rhel-0 ~]# cat /etc/containers/registries.conf 
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"]

[[registry]]
  location = "quay.io/openshift-release-dev/ocp-v4.0-art-dev"
  insecure = false
  blocked = false
  mirror-by-digest-only = true
  prefix = ""

  [[registry.mirror]]
    location = "internal-registry.qe.devcluster.openshift.com:5000/ocp/release"
    insecure = false

[[registry]]
  location = "registry.svc.ci.openshift.org/ocp/release"
  insecure = false
  blocked = false
  mirror-by-digest-only = true
  prefix = ""

  [[registry.mirror]]
    location = "internal-registry.qe.devcluster.openshift.com:5000/ocp/release"
    insecure = false


TASK [openshift_node : Pull release image] *************************************
Friday 30 August 2019  14:02:29 +0800 (0:00:00.734)       0:03:17.724 ********* 
changed: [rhel-0.qe-jialiu2.qe.devcluster.openshift.com] => {"changed": true, "cmd": ["podman", "pull", "--tls-verify=False", "--authfile", "/tmp/ansible.UNT3I2/pull-secret.json", "registry.svc.ci.openshift.org/ocp/release@sha256:7319aa4ba33821a78710b27f9f4bbef68daee41ab3e9a7286976f0d2e050a85b"], "delta": "0:00:10.387206", "end": "2019-08-30 02:04:03.782388", "rc": 0, "start": "2019-08-30 02:03:53.395182", "stderr": "Trying to pull registry.svc.ci.openshift.org/ocp/release@sha256:7319aa4ba33821a78710b27f9f4bbef68daee41ab3e9a7286976f0d2e050a85b...Getting image source signatures\nCopying blob sha256:79d11c1a86c4521558eb9e4b4173811ad15cd2c865eda4eb15a1a2e90af94136\nCopying blob sha256:c8d67acdb2ffaebd638cf55a8fccc63693211060670aa7f0ea1d65b5d2c674dd\nCopying blob sha256:dfb1ce7ba7ceca0e20a9b941100a2dba37df67ba72d55b85c562f40d3be43aba\nCopying blob sha256:e58465c4bf5189edff71a713c052c1d318b1aa6e14b9cf7246ee39aa41beac37\nCopying blob sha256:92a308e38576b60f4d71075d621215a51608bf29e6814896828b2936b5ac936c\nCopying blob sha256:7f7a258f73628b59a5dca7eb8cfc984b6d02f2f70771d046440b0423dffb1d57\nCopying config sha256:20268cd04d0956e66d072ddf879450d6b382529671fef09992a8fa95fc92e731\nWriting manifest to image destination\nStoring signatures", "stderr_lines": ["Trying to pull registry.svc.ci.openshift.org/ocp/release@sha256:7319aa4ba33821a78710b27f9f4bbef68daee41ab3e9a7286976f0d2e050a85b...Getting image source signatures", "Copying blob sha256:79d11c1a86c4521558eb9e4b4173811ad15cd2c865eda4eb15a1a2e90af94136", "Copying blob sha256:c8d67acdb2ffaebd638cf55a8fccc63693211060670aa7f0ea1d65b5d2c674dd", "Copying blob sha256:dfb1ce7ba7ceca0e20a9b941100a2dba37df67ba72d55b85c562f40d3be43aba", "Copying blob sha256:e58465c4bf5189edff71a713c052c1d318b1aa6e14b9cf7246ee39aa41beac37", "Copying blob sha256:92a308e38576b60f4d71075d621215a51608bf29e6814896828b2936b5ac936c", "Copying blob sha256:7f7a258f73628b59a5dca7eb8cfc984b6d02f2f70771d046440b0423dffb1d57", "Copying config sha256:20268cd04d0956e66d072ddf879450d6b382529671fef09992a8fa95fc92e731", "Writing manifest to image destination", "Storing signatures"], "stdout": "20268cd04d0956e66d072ddf879450d6b382529671fef09992a8fa95fc92e731", "stdout_lines": ["20268cd04d0956e66d072ddf879450d6b382529671fef09992a8fa95fc92e731"]}

Comment 4 errata-xmlrpc 2019-10-16 06:37:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922


Note You need to log in before you can comment on or make changes to this bug.