Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1821490

Summary:

[OSP16] undercloud install fails with error message "Paunch failed with config_id tripleo_step4"

Product:

Red Hat OpenStack

Reporter:

ctrinh

Component:

openstack-heat-templates

Assignee:

Steve Baker <sbaker>

Status:

CLOSED DUPLICATE

QA Contact:

Arik Chernetsky <achernet>

Severity:

high

Docs Contact:

Priority:

medium

Version:

16.0 (Train)

CC:

apetrich, apevec, aschultz, emacchi, lhh

Target Milestone:

---

Keywords:

Triaged

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2020-04-21 20:06:13 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
undercloud-install-20200406180453.tar.bzip2	none

Description ctrinh 2020-04-06 23:10:46 UTC

Created attachment 1676712 [details]
undercloud-install-20200406180453.tar.bzip2

Created attachment 1676712 [details]
undercloud-install-20200406180453.tar.bzip2

I'm attempting to setup an undercloud on my RHEL 8.1 VM and repeatedly run into this error "Paunch failed with config_id tripleo_step4"

I see some 401 errors but I do have the proper credentials set to access registry.redhat.io

undercloud install log is attached. sosreport can be downloaded here:
https://ctrinh-rhel-logs.s3-us-west-1.amazonaws.com/sosreport-ctrinh-rhel01-1821430-2020-04-07-jwoobju.tar.xz

*******************************************
2020-04-06 20:27:26,779 p=513724 u=root |  Monday 06 April 2020  20:27:26 +0300 (0:00:00.443)       0:03:22.239 **********
2020-04-06 20:27:27,391 p=513724 u=root |  changed: [ctrinh-rhel01]
2020-04-06 20:27:27,520 p=513724 u=root |  TASK [tripleo-container-image-prepare : Run tripleo-container-image-prepare logged to: /var/log/tripleo-container-image-prepare.log] ***********************************************************************
2020-04-06 20:27:27,521 p=513724 u=root |  Monday 06 April 2020  20:27:27 +0300 (0:00:00.741)       0:03:22.980 **********
2020-04-06 21:04:52,091 p=513724 u=root |  fatal: [ctrinh-rhel01]: FAILED! => {"changed": true, "cmd": ["/usr/bin/tripleo-container-image-prepare", "--roles-file", "/tmp/ansible.mcy9wf5q-role-data", "--environment-file", "/tmp/ansible.weqvtvl5-prepare-param", "--cleanup", "partial", "--log-file", "/var/log/tripleo-container-image-prepare.log", "--debug"], "delta": "0:37:24.126746", "end": "2020-04-06 21:04:51.902821", "msg": "non-zero return code", "rc": 1, "start": "2020-04-06 20:27:27.776075", "stderr": "DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2/ HTTP/1.1\" 401 99\nDEBUG:tripleo_common.image.image_uploader:https://registry.redhat.io/v2/ status code 401\nDEBUG:tripleo_common.image.image_uploader:Token parameters: params {'service': 'docker-registry', 'scope': 'repository:rhosp-rhel8/openstack-ironic-inspector:pull'}\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): registry.redhat.io:443\nDEBUG:urllib3.connectionpool:https://registry.redhat.io:443 \"GET /v2 HTTP/1.1\" 401 99\nDEBUG:urllib3.util.retry
...

Comment 1 Alex Schultz 2020-04-16 14:40:54 UTC

The logs provided don't seem to getting to step4 as indicated in the bug report. So it's failing on authentication when trying to fetch the containers. Have you tried double checking that the provided credentials are valid by testing with a 'podman login'?  Additionally there was a bug where if the bandwidth was too low, it would throw an error while fetching the containers.  We resolved that via Bug 1813520, which hasn't been shipped yet. I think you might be hitting that bug.

Comment 2 ctrinh 2020-04-16 20:35:06 UTC

Yes, I have checked that my credentials work. I went through the steps here (https://access.redhat.com/articles/3560571) to ensure I could access the registry.

Okay I will look to try that fix when it is delivered.

[root@ctrinh-rhel01 ~]# podman login registry.redhat.io
Username: 7848345|infi-service-dev-acct
Password:
Login Succeeded!
[root@ctrinh-rhel01 ~]# podman image list
REPOSITORY                                                          TAG        IMAGE ID       CREATED       SIZE
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-server              16.0-94    61832761b47a   13 days ago   1.04 GB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-api                 16.0-93    5c3ec04bfc46   13 days ago   1.13 GB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-l3-agent            16.0-95    e101057c08b0   13 days ago   1.07 GB
192.168.24.1:8787/rhosp-rhel8/openstack-zaqar-wsgi                  16.0-91    5b27082ef8f8   13 days ago   738 MB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-api                    16.0-94    14dc25e3d744   13 days ago   1.15 GB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-object                16.0-94    1818e4c59199   13 days ago   789 MB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent   16.0-94    6643faa5ee42   13 days ago   926 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-conductor            16.0-91    d458bb91f4dc   13 days ago   932 MB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-scheduler              16.0-93    8ebbce44d268   13 days ago   1.23 GB
192.168.24.1:8787/rhosp-rhel8/openstack-keystone                    16.0-93    8c85b92f2358   13 days ago   787 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-proxy-server          16.0-89    39c93e1df535   13 days ago   836 MB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-dhcp-agent          16.0-94    29dfdcce3e9a   13 days ago   1.07 GB
192.168.24.1:8787/rhosp-rhel8/openstack-glance-api                  16.0-91    220b82cf191d   13 days ago   1.04 GB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-engine              16.0-93    d79ee2d20d73   13 days ago   1.11 GB
192.168.24.1:8787/rhosp-rhel8/openstack-heat-engine                 16.0-91    852fbca8d16d   13 days ago   895 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-account               16.0-94    3d1b34eb956d   13 days ago   789 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-pxe                  16.0-91    a675caedbcd3   13 days ago   815 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-container             16.0-93    f7eb6c7d88e8   13 days ago   789 MB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-executor            16.0-105   0617ff214948   13 days ago   1.51 GB
192.168.24.1:8787/rhosp-rhel8/openstack-heat-api                    16.0-91    3db4c8ef499c   13 days ago   895 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-api                  16.0-91    567e149f8247   13 days ago   810 MB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-compute-ironic         16.0-91    b608911e0548   13 days ago   2.01 GB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-neutron-agent        16.0-96    c28b607ba6c9   13 days ago   926 MB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-conductor              16.0-93    90fe4bb79351   13 days ago   1.05 GB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-event-engine        16.0-97    221f0e4000bd   13 days ago   1.11 GB
192.168.24.1:8787/rhosp-rhel8/openstack-placement-api               16.0-95    2a3d3ce2df47   13 days ago   663 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-inspector            16.0-95    09b211d63b47   13 days ago   711 MB
192.168.24.1:8787/rhosp-rhel8/openstack-tempest                     16.0-93    cd1c5d2464a6   13 days ago   944 MB
192.168.24.1:8787/rhosp-rhel8/openstack-keepalived                  16.0-97    33713328ab9b   13 days ago   449 MB
192.168.24.1:8787/rhosp-rhel8/openstack-haproxy                     16.0-101   3b3a5ff299d2   13 days ago   573 MB
192.168.24.1:8787/rhosp-rhel8/openstack-rabbitmq                    16.0-101   e9b6eed29224   13 days ago   616 MB
192.168.24.1:8787/rhosp-rhel8/openstack-cron                        16.0-96    180cbd92d9f4   13 days ago   434 MB
192.168.24.1:8787/rhosp-rhel8/openstack-memcached                   16.0-99    a5291b73a22d   13 days ago   455 MB
192.168.24.1:8787/rhosp-rhel8/openstack-iscsid                      16.0-98    6ffc378bfccd   13 days ago   454 MB
192.168.24.1:8787/rhosp-rhel8/openstack-mariadb                     16.0-101   01cbbfc926c6   13 days ago   787 MB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-server              16.0-81    6d0694a95668   7 weeks ago   1.04 GB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-api                 16.0-81    8c40bb592ba8   7 weeks ago   1.13 GB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-l3-agent            16.0-82    ceb6d3d9ea5c   7 weeks ago   1.07 GB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent   16.0-81    394cfe63ee5e   7 weeks ago   925 MB
192.168.24.1:8787/rhosp-rhel8/openstack-zaqar-wsgi                  16.0-79    0bc144e9c690   7 weeks ago   737 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-object                16.0-82    f2cf30a04b40   7 weeks ago   788 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-conductor            16.0-80    2dd011788eed   7 weeks ago   931 MB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-scheduler              16.0-82    149794501fb3   7 weeks ago   1.23 GB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-api                    16.0-83    3e6638c90131   7 weeks ago   1.15 GB
192.168.24.1:8787/rhosp-rhel8/openstack-keystone                    16.0-82    2d39eacbbf4a   7 weeks ago   786 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-proxy-server          16.0-78    70fa70339d02   7 weeks ago   836 MB
192.168.24.1:8787/rhosp-rhel8/openstack-neutron-dhcp-agent          16.0-81    027dab4e0e8f   7 weeks ago   1.07 GB
192.168.24.1:8787/rhosp-rhel8/openstack-glance-api                  16.0-80    5a43edc85fec   7 weeks ago   1.04 GB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-engine              16.0-81    3666beb77613   7 weeks ago   1.11 GB
192.168.24.1:8787/rhosp-rhel8/openstack-heat-engine                 16.0-80    b8865a7726e6   7 weeks ago   894 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-account               16.0-82    f9c49b420b87   7 weeks ago   788 MB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-executor            16.0-93    e91272eaf36f   7 weeks ago   1.51 GB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-pxe                  16.0-80    c5930e0e0457   7 weeks ago   814 MB
192.168.24.1:8787/rhosp-rhel8/openstack-swift-container             16.0-82    30f1b8861344   7 weeks ago   788 MB
192.168.24.1:8787/rhosp-rhel8/openstack-heat-api                    16.0-80    65f9fa38acf0   7 weeks ago   894 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-neutron-agent        16.0-83    6e9a16c4933a   7 weeks ago   925 MB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-api                  16.0-80    a82f36768cfd   7 weeks ago   809 MB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-conductor              16.0-82    f65a909e334d   7 weeks ago   1.05 GB
192.168.24.1:8787/rhosp-rhel8/openstack-nova-compute-ironic         16.0-80    34b04b138aa9   7 weeks ago   2.01 GB
192.168.24.1:8787/rhosp-rhel8/openstack-mistral-event-engine        16.0-85    aca0d7ee2a3f   7 weeks ago   1.11 GB
192.168.24.1:8787/rhosp-rhel8/openstack-ironic-inspector            16.0-83    0cc2092d93ad   7 weeks ago   711 MB
192.168.24.1:8787/rhosp-rhel8/openstack-placement-api               16.0-82    e1cf4edb9996   7 weeks ago   663 MB
192.168.24.1:8787/rhosp-rhel8/openstack-tempest                     16.0-81    681dc5037699   7 weeks ago   943 MB
192.168.24.1:8787/rhosp-rhel8/openstack-keepalived                  16.0-86    a33927990cc5   7 weeks ago   448 MB
192.168.24.1:8787/rhosp-rhel8/openstack-rabbitmq                    16.0-89    c40afeea817b   7 weeks ago   616 MB
192.168.24.1:8787/rhosp-rhel8/openstack-haproxy                     16.0-88    30653ad92cb4   7 weeks ago   572 MB
192.168.24.1:8787/rhosp-rhel8/openstack-memcached                   16.0-88    768efc2e35c2   7 weeks ago   455 MB
192.168.24.1:8787/rhosp-rhel8/openstack-cron                        16.0-85    60009a28824e   7 weeks ago   434 MB
192.168.24.1:8787/rhosp-rhel8/openstack-iscsid                      16.0-87    eedf7243b5d0   7 weeks ago   453 MB
192.168.24.1:8787/rhosp-rhel8/openstack-mariadb                     16.0-90    8d5d9ee4b240   7 weeks ago   787 MB
[root@ctrinh-rhel01 ~]#

Comment 3 Emilien Macchi 2020-04-21 20:06:13 UTC


*** This bug has been marked as a duplicate of bug 1813520 ***