Doing Upgrading the Undercloud Node is failing with error messages ~~~ [root@dir ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.4 (Maipo) [root@dir ~]# ~~~ ~~~ undercloud.conf ~~~ ~~~ [DEFAULT] local_interface = eth1 local_ip = 192.168.24.1/24 network_gateway = 192.168.24.1 undercloud_public_vip = 192.168.24.2 undercloud_admin_vip = 192.168.24.3 network_cidr = 192.168.24.0/24 masquerade_network = 192.168.24.0/24 dhcp_start = 192.168.24.5 dhcp_end = 192.168.24.24 inspection_iprange = 192.168.24.100,192.168.24.120 generate_service_certificate = true certificate_generation_ca = local ~~~ ~~~ 2018-07-28 15:17:27,854 INFO: Created flavor "block-storage" with profile "block-storage" 2018-07-28 15:17:28,008 INFO: Created flavor "swift-storage" with profile "swift-storage" 2018-07-28 15:18:26,895 INFO: ############################################################################# Undercloud install complete. The file containing this installation's passwords is at /home/stack/undercloud-passwords.conf. There is also a stackrc file at /home/stack/stackrc. These files are needed to interact with the OpenStack services, and should be secured. ############################################################################# [stack@dir ~]$ ~~~ ~~~ [stack@dir ~]$ openstack catalog list +------------------+-------------------------+----------------------------------------------------------------------------------+ | Name | Type | Endpoints | +------------------+-------------------------+----------------------------------------------------------------------------------+ | gnocchi | metric | regionOne | | | | publicURL: https://192.168.24.2:13041 | | | | internalURL: http://192.168.24.3:8041 | | | | adminURL: http://192.168.24.3:8041 | | | | | | placement | placement | regionOne | | | | publicURL: https://192.168.24.2:13778/placement | | | | internalURL: http://192.168.24.3:8778/placement | | | | adminURL: http://192.168.24.3:8778/placement | | | | | | neutron | network | regionOne | | | | publicURL: https://192.168.24.2:13696 | | | | internalURL: http://192.168.24.3:9696 | | | | adminURL: http://192.168.24.3:9696 | | | | | | aodh | alarming | regionOne | | | | publicURL: https://192.168.24.2:13042 | | | | internalURL: http://192.168.24.3:8042 | | | | adminURL: http://192.168.24.3:8042 | | | | | | glance | image | regionOne | | | | publicURL: https://192.168.24.2:13292 | | | | internalURL: http://192.168.24.3:9292 | | | | adminURL: http://192.168.24.3:9292 | | | | | | ceilometer | metering | regionOne | | | | publicURL: https://192.168.24.2:13777 | | | | internalURL: http://192.168.24.3:8777 | | | | adminURL: http://192.168.24.3:8777 | | | | | | heat-cfn | cloudformation | regionOne | | | | publicURL: https://192.168.24.2:13800/v1/0b1e4fd331804d50b64e543ae8733b5d | | | | internalURL: http://192.168.24.3:8000/v1/0b1e4fd331804d50b64e543ae8733b5d | | | | adminURL: http://192.168.24.3:8000/v1/0b1e4fd331804d50b64e543ae8733b5d | | | | | | ironic | baremetal | regionOne | | | | publicURL: https://192.168.24.2:13385 | | | | internalURL: http://192.168.24.3:6385 | | | | adminURL: http://192.168.24.3:6385 | | | | | | nova | compute | regionOne | | | | publicURL: https://192.168.24.2:13774/v2.1 | | | | internalURL: http://192.168.24.3:8774/v2.1 | | | | adminURL: http://192.168.24.3:8774/v2.1 | | | | | | zaqar-websocket | messaging-websocket | regionOne | | | | publicURL: wss://192.168.24.2:9000 | | | | internalURL: ws://192.168.24.3:9000 | | | | adminURL: ws://192.168.24.3:9000 | | | | | | heat | orchestration | regionOne | | | | publicURL: https://192.168.24.2:13004/v1/0b1e4fd331804d50b64e543ae8733b5d | | | | internalURL: http://192.168.24.3:8004/v1/0b1e4fd331804d50b64e543ae8733b5d | | | | adminURL: http://192.168.24.3:8004/v1/0b1e4fd331804d50b64e543ae8733b5d | | | | | | mistral | workflowv2 | regionOne | | | | publicURL: https://192.168.24.2:13989/v2 | | | | internalURL: http://192.168.24.3:8989/v2 | | | | adminURL: http://192.168.24.3:8989/v2 | | | | | | swift | object-store | regionOne | | | | publicURL: https://192.168.24.2:13808/v1/AUTH_0b1e4fd331804d50b64e543ae8733b5d | | | | internalURL: http://192.168.24.3:8080/v1/AUTH_0b1e4fd331804d50b64e543ae8733b5d | | | | adminURL: http://192.168.24.3:8080 | | | | | | zaqar | messaging | regionOne | | | | publicURL: https://192.168.24.2:13888 | | | | internalURL: http://192.168.24.3:8888 | | | | adminURL: http://192.168.24.3:8888 | | | | | | ironic-inspector | baremetal-introspection | regionOne | | | | publicURL: https://192.168.24.2:13050 | | | | internalURL: http://192.168.24.3:5050 | | | | adminURL: http://192.168.24.3:5050 | | | | | | panko | event | regionOne | | | | publicURL: https://192.168.24.2:13779 | | | | internalURL: http://192.168.24.3:8779 | | | | adminURL: http://192.168.24.3:8779 | | | | | | keystone | identity | regionOne | | | | publicURL: https://192.168.24.2:13000/v2.0 | | | | internalURL: http://192.168.24.3:5000/v2.0 | | | | adminURL: http://192.168.24.3:35357/v2.0 | | | | | +------------------+-------------------------+----------------------------------------------------------------------------------+ [stack@dir ~]$ ~~~ ++++++++++++ MINOR UPDATE ++++++++++++ ~~~ 2018-07-28 15:33:05,732 INFO: Not creating default plan "overcloud" because it already exists. 2018-07-28 15:33:06,769 INFO: ############################################################################# Undercloud upgrade complete. The file containing this installation's passwords is at /home/stack/undercloud-passwords.conf. There is also a stackrc file at /home/stack/stackrc. These files are needed to interact with the OpenStack services, and should be secured. ############################################################################# [stack@dir ~]$ ~~~ [stack@dir ~]$ [stack@dir ~]$ [stack@dir ~]$ sudo openssl x509 -text -noout -in /etc/pki/tls/certs/undercloud-192.168.24.2.pem | grep -i Alter -C 4 f7:c0:61:dd:42:91:56:61:31:b6:dd:27:98:ff:a1: 51:eb Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Alternative Name: DNS:192.168.24.2 X509v3 Extended Key Usage: TLS Web Client Authentication, TLS Web Server Authentication X509v3 Basic Constraints: critical [stack@dir ~]$ [stack@dir ~]$ rpm -qi puppet-certmonger-1.1.1-1.1157a7egit.el7ost.noarch Name : puppet-certmonger Version : 1.1.1 Release : 1.1157a7egit.el7ost Architecture: noarch Install Date: Sat 28 Jul 2018 02:28:13 PM EDT Group : Unspecified Size : 49470 License : Apache-2.0 Signature : RSA/SHA256, Tue 29 Nov 2016 12:16:00 PM EST, Key ID 199e2f91fd431d51 Source RPM : puppet-certmonger-1.1.1-1.1157a7egit.el7ost.src.rpm Build Date : Thu 27 Oct 2016 03:52:35 PM EDT Build Host : x86-038.build.eng.bos.redhat.com Relocations : (not relocatable) Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla> Vendor : Red Hat, Inc. URL : https://github.com/earsdown/puppet-certmonger Summary : Certmonger Puppet Module Description : Certmonger puppet module for integration with IPA CAs. [stack@dir ~]$ ++++++++++++ MAJOR UPDATE ++++++++++++ * While following https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html-single/upgrading_red_hat_openstack_platform/#upgrading_the_undercloud_node * Run the following command to upgrade the undercloud: $ openstack undercloud upgrade * Is failing with below error messages, ~~~ 2018-07-28 16:23:51,252 INFO: Error: /Stage[main]/Heat::Keystone::Domain/Keystone_user[heat_admin::heat_stack]: Could not evaluate: Execution of '/bin/openstack token issue --format value' returned 1: Certificate did not match expected hostname: 192.168.24.2. Certificate: {'subjectAltName': [('DNS', '192.168.24.2')], 'subject': ((('commonName', u'192.168.24.2'),),)} 2018-07-28 16:23:51,252 INFO: SSL exception connecting to https://192.168.24.2:13000/v3/auth/tokens: hostname '192.168.24.2' doesn't match '192.168.24.2' (tried 44, for a total of 170 seconds) 2018-07-28 16:23:51,253 INFO: Notice: /Stage[main]/Heat::Keystone::Domain/Keystone_user_role[heat_admin::heat_stack@::heat_stack]: Dependency Keystone_user[heat_admin::heat_stack] has failures: true 2018-07-28 16:23:51,253 INFO: Warning: /Stage[main]/Heat::Keystone::Domain/Keystone_user_role[heat_admin::heat_stack@::heat_stack]: Skipping because of failed dependencies 2018-07-28 16:23:53,575 INFO: Notice: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_endpoint[regionOne/keystone::identity]/public_url: public_url changed 'https://192.168.24.2:13000/v2.0' to 'https://192.168.24.2:13000' 2018-07-28 16:23:53,576 INFO: Notice: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_endpoint[regionOne/keystone::identity]/internal_url: internal_url changed 'http://192.168.24.3:5000/v2.0' to 'http://192.168.24.3:5000' 2018-07-28 16:23:53,576 INFO: Notice: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_endpoint[regionOne/keystone::identity]/admin_url: admin_url changed 'http://192.168.24.3:35357/v2.0' to 'http://192.168.24.3:35357' ~~~
This requires a newer version of puppet-certmonger. The one that's being used there has a bug where it requests the certificate with the erroneous subjectAltName type (DNS always, and not IP). Having done this, you need to remove the certificate (and the certificate request) from the undercloud, and try again the undercloud install. It should do a new request with the correct parameters.
We need to backport these RDO change for the downstream side, also we need to check how RDO are building certmonger to see what is necessary to change the downstream as well.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:3587